Difference between revisions of "MogileFS setup"

From Dreamwidth Notes
Jump to: navigation, search
(Install Prerequisites)
(Configuring MogileFS for a Dreamwidth development installation)
Line 15: Line 15:
 
=== Install Prerequisites ===
 
=== Install Prerequisites ===
  
On your Dreamwidth machine, which we assume is running Ubuntu 9.04 (jaunty), you need to make sure the file <tt>/etc/apt/sources.list</tt> contains the following section:
+
OBSOLETE INSTRUCTIONS. These need to be looked at.  If you really need MogileFS for something, please talk to us and we'll see about getting this setup.
 
+
  # MogileFS, Perlbal, related packages
+
deb http://ppa.launchpad.net/yola/ppa/ubuntu hardy main
+
deb-src http://ppa.launchpad.net/yola/ppa/ubuntu hardy main
+
 
+
If that section does not exist, add it at the bottom.  Now save and exit the file and run the following commands:
+
 
+
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3CEEB082
+
sudo apt-get update
+
 
+
And now you can install everything:
+
 
+
sudo apt-get install mogilefs-server mogilefs-utils
+
 
+
You will now need to configure your installation.
+
 
+
=== Build and Install MogileFS Debian packages ===
+
 
+
You should already have the source available from the scratch install.  So
+
 
+
  $ cd $LJHOME/cvs/mogilefs
+
 
+
Now you have to do a two-phase build and install:
+
 
+
  $ bin/build-all-debian.sh
+
  $ sudo dpkg -i ./api/perl/libmogilefs-perl_1.00-1_all.deb
+
  $ bin/build-all-debian.sh
+
  $ sudo dpkg -i packages/mogilefsd_1.00-2_all.deb packages/mogilefs-utils_0.01-1_all.deb packages/mogstored_1.00-2_all.deb
+
  
 
=== Create MogileFS database ===
 
=== Create MogileFS database ===

Revision as of 01:28, 24 March 2010

Note: This page is still a draft. Much of this information was pulled from the MogileFS Wiki combined with the INSTALL.txt instructions from the mogilefs source.

Configuring MogileFS for a Dreamwidth development installation

These are instructions for getting a simple MogileFS installation working on a single instance (likely development/testing) Dreamwidth installation. This is not a full setup document for a production, multi-server environment.

Basic Steps

  • Install prerequisites
  • Build and install MogileFS Debian packages
  • Create MogileFS Database
  • Configure MogileFS servers
  • Set up Dreamwidth installation to work with MogileFS

Install Prerequisites

OBSOLETE INSTRUCTIONS. These need to be looked at. If you really need MogileFS for something, please talk to us and we'll see about getting this setup.

Create MogileFS database

You should already have MySQL installed for your Dreamwidth installation. So log in as root there and create the mogilefs user:

$ mysql -u root -p
mysql> create database mogilefs;
mysql> grant all on mogilefs.* to 'mogile'@'%' identified by 'mogilepw';
mysql> flush privileges;
mysql> quit

Then go ahead and create the schema using mogdbsetup

$ /usr/bin/mogdbsetup --yes --dbname=mogilefs --dbuser=mogile --dbpassword=mogilepw

Configure MogileFS servers

You will need to set up at least one tracker (mogilefsd) and one storage server (mogstored), but ideally several of each on different machines for redundancy. Keep in mind this is a simple development setup.

The tracker can be configured with /etc/mogilefs/mogilefsd.conf as follows:

$ cat > /etc/mogilefs/mogilefsd.conf <<EOF
daemonize = 1
db_dsn = DBI:mysql:mogilefs
db_user = mogile
db_pass = mogilepw
listen = 127.0.0.1:7001
conf_port = 7001
default_mindevcount = 1
EOF

Then start the tracker:

$ /usr/bin/mogilefsd

The storage server can be configured with /etc/mogilefs/mogstored.conf as follows:

$ cat > /etc/mogilefs/mogstored.conf <<EOF
daemonize = 1
httplisten = 127.0.0.1:7500
mgmtlisten = 127.0.0.1:7501
docroot = /var/mogdata
EOF
$ mkdir /var/mogdata

Then start the storage server:

$ /usr/bin/mogstored

Next you need to pass some configuration info to the tracker with mogadm, but mogadm needs a config file too:

$ cat > /etc/mogilefs/mogilefs.conf <<EOF
trackers = 127.0.0.1:7001
EOF
$ /usr/bin/mogadm host add localhost --ip=127.0.0.1 --port=7500 --status=alive
$ /usr/bin/mogadm device add localhost 1
$ mkdir /var/mogdata/dev1

Create a MogileFS domain and storage classes to be used by Dreamwidth:

$ /usr/bin/mogadm domain add dreamwidth
$ /usr/bin/mogadm class add dreamwidth temp --mindevcount=1
$ /usr/bin/mogadm class add dreamwidth captcha --mindevcount=1
$ /usr/bin/mogadm class add dreamwidth userpics --mindevcount=1

Now you have a working MogileFS installation that you can test with mogtool:

$ cat > /etc/mogilefs/mogtool.conf <<EOF
trackers = 127.0.0.1:7001
domain = dreamwidth
class = temp
EOF
$ /usr/bin/mogtool inject /etc/hosts hosts
$ /usr/bin/mogtool extract hosts -
$ /usr/bin/mogtool delete hosts

Set up Dreamwidth installation to work with MogileFS

Add to $LJHOME/etc/config-local.pl:

   %MOGILEFS_CONFIG = (
       hosts => [ '127.0.0.1:7001', ],
       domain => 'dreamwidth',
   );
   $USERPIC_MOGILEFS = 1;
   $CAPTCHA_MOGILEFS = 1;

Restart Apache, and try uploading an icon via editpics.bml