Dreamwidth Scratch Installation

Firstly, you'll need to set up a Linux system -- here are the Suggested Server Requirements. These instructions assume Ubuntu 12.04 or up. Please let us know if you run into difficulties--we want to update this if people are having problems! We recommend coming into the #dreamwidth-dev channel on IRC.

After you set up, you can keep updated with the instructions on Dev Maintenance.

If you have difficulties or problems or suggestions for installation procedures, please report them to Bug 61. Also look at Production Setup: Webserver.

Bringing the system up to date
Before you start installing anything else, you might want to make sure your system is up to date. You should update the packages on your system (either in a root shell (sudo bash) or preceding these commands with 'sudo'):

apt-get update apt-get upgrade

If you want man pages on your system:

apt-get install man

Installing necessary packages
This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands.

apt-get install git-core subversion apache2-mpm-prefork \ libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \ mysql-server wget unzip links vim libclass-autouse-perl \ libdatetime-perl libcache-memcached-perl libhash-multivalue-perl \ libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \ libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \ libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \ libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \ libgnupg-interface-perl libmail-gnupg-perl perlmagick \ libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \ libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \ libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \ libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \ libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \ libtest-simpleunit-perl libtemplate-perl libterm-readkey-perl \ gcc libtest-most-perl libgearman-client-perl \ libbusiness-creditcard-perl liblwpx-paranoidagent-perl \ libtheschwartz-perl libfile-type-perl libjson-perl

This will download about 113MB of files and use around 355MB of disk space.

Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again.

Then
Check whether your system is using threaded MPMs or pre-fork. You will need the latter. (Note: if you ran the above command to install Apache, then you have the pre-fork version installed.)

If you have weird errors saying things like "Package mercurial is not available, but is referred to by another package.", try editing /etc/apt/sources.list and uncommenting the other repositories and then doing an apt-get update. Ran into this problem on a fresh install of jaunty on Linode.

You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay.

Before installing, check if your CPAN shell has 'make' at the correct location. Run which make, then compare the results with running o conf make at the CPAN shell (to run the CPAN shell, type cpan.)

The perl libraries are: cpan Bundle::CPAN  # this will guide you through setting up CPAN cpan GTop cpan Digest::SHA1 cpan Unicode::CheckUTF8 cpan MogileFS::Client  # this is necessary even if you don't use MogileFS cpan TheSchwartz::Worker::SendEmail cpan TheSchwartz::Worker::PubSubHubbubPublish cpan Digest::SHA256 cpan Text::Markdown

See Bundle::CPAN if you are having trouble installing Bundle::CPAN

You will also want to install and configure Postfix so your DW can send out email:

apt-get install postfix

If you choose not to configure on the install, you can do so later with:

dpkg-reconfigure postfix

Set Timezone to UTC
Dreamwidth will complain if you aren't running in UTC. If you're not configured for UTC already, you can reset it (on Ubuntu/Debian) using

dpkg-reconfigure tzdata

and choose Etc/UTC.

Setting up the DW user account
Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files.

To set up the user account:

adduser username

Fill out the user's info and put them in the sudo group:

usermod -a -G sudo username

So, to use  as a username:

adduser dw usermod -a -G sudo dw

Now, we will get the $LJHOME variable set. Log into your user:

su - dw

We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.

For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file:

export LJHOME=/home/dw/dw

Or wherever it is that you want the root of your source code install to be. Yes, the variable is named LJHOME. We'll live with it for now. ;-)

Test this. Log out of the dw user account and log back in, then type:

echo $LJHOME

You should see /home/dw/dw or whatever you set it to. If you don't, then something has gone awry -- maybe the export line isn't in the right place..?

Set up GitHub
First, create an account on Github. Then, you will need to fork these two repositories:


 * https://github.com/dreamwidth/dw-free
 * https://github.com/dreamwidth/dw-nonfree

Go to each of their pages, and click the "Fork" button. This sets up your own copy of Dreamwidth's code which use to make and submit your changes.

Here's a quick overview of how the repositories will work together:


 * dw-free is the main repository and goes into $LJHOME
 * other repositories will go into $LJHOME/ext
 * personal config files go into $LJHOME/ext/local
 * All code from dw-free ($LJHOME) and the additional repos under $LJHOME/ext are automatically live when you start your server. There's no need to run any additional syncing steps anymore.

Set up dw-free
Replace the USERNAME sections of the URL with your Github username. This will make your repository an authenticated version that can push changes back to Github, as opposed to only being able to pull them.

cd ~/ git clone https://USERNAME@github.com/USERNAME/dw-free.git $LJHOME cd $LJHOME git remote add dreamwidth https://github.com/dreamwidth/dw-free git fetch dreamwidth git branch --set-upstream develop dreamwidth/develop git branch --set-upstream master dreamwidth/master
 * 1) clone a copy of the repository onto your machine
 * 1) and let's make it aware of the dreamwidth repository so we can grab updates later
 * 1) we also want to sync up with the main branches

Set up dw-nonfree
cd $LJHOME/ext git clone https://USERNAME@github.com/USERNAME/dw-nonfree.git cd dw-nonfree git remote add dreamwidth https://github.com/dreamwidth/dw-nonfree git fetch dreamwidth git branch --set-upstream develop dreamwidth/develop git branch --set-upstream master dreamwidth/master cd ..

Database setup
You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root:

mysql -u root -p

These MySQL commands wil create your development database:

create database dw; grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';

You might want to pick a more appropriate database username/database name/password.

While you're here, you probably want to go ahead and create a database for TheSchwartz, too, unless you really know that you won't need it:

create database dw_schwartz; grant all on dw_schwartz.* to 'dw'@'localhost' identified by 'somePassword';

When you're done, exit out of mysql:

quit;

Editing the config files
Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now:

cd $LJHOME mkdir -p ext/local/etc echo "highest" > ext/local/.dir_scope cp doc/config-local.pl.txt ext/local/etc/config-local.pl vim ext/local/etc/config-local.pl
 * 1) copy over your config files, and tell them to take priority
 * 2) over config files in any of the repos

Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation.

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your $DOMAIN and the %DBINFO structure.

cd $LJHOME cp doc/config-private.pl.txt ext/local/etc/config-private.pl chmod go-rwx ext/local/etc/config-private.pl vim ext/local/etc/config-private.pl

In this file, uncomment the lines in the %DBINFO blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example:

%DBINFO = (       master => {            pass => 'mypassword',        },    );

Note that there are two DBINFO blocks; get them both.

If you created the database for TheSchwartz earlier, you'll also want to configure it:

# Schwartz DB configuration @THESCHWARTZ_DBS = (           {                dsn => 'dbi:mysql:dw_schwartz;host=localhost',                user => 'dw',                pass => '',     # CHANGETHIS            },        );

The base configuration file is under source control, and is already in etc/config.pl</tt>. You should not need to change anything very much in this, but you might have to change some stuff. If you find you do have to touch this, copy it over first, then you can edit it:

cd $LJHOME cp etc/config.pl ext/local/etc/config.pl vim ext/local/etc/config.pl

Note that from here on, when any documentation says to edit etc/config*, you'll likely want to edit ext/local/etc/config-* instead.

Make sure things are working with checkconfig.pl
Now, you need to see if everything is working.

If you've setup the files as indicated above, run this command:

cd $LJHOME bin/checkconfig.pl --no=ljconfig

If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC, the mailing list, or make a post to dw_dev.

Populate database with initial data
There are a few commands you can now run to install the database. Just run these and watch for errors.

Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors.

$LJHOME/bin/upgrading/update-db.pl -r --innodb $LJHOME/bin/upgrading/update-db.pl -r --innodb # at least for now we have to run this twice $LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb $LJHOME/bin/upgrading/update-db.pl -p

If you created your database for TheSchwartz earlier, you'll also want to populate it with its default tables:

mysql -u dw -p dw_schwartz < /usr/share/doc/libtheschwartz-perl/schema.sql

Create a system account:

$LJHOME/bin/upgrading/make_system.pl

(That step will ask you for a password for the System account. You can change it later by logging in as system, so just give it something for now.)

And load in the translation strings:

$LJHOME/bin/upgrading/texttool.pl load

Configure Apache
This step will need to be done as the root user. Below is the Apache 2 configuration running on the Dreamwidth staging site. Put this in a file named "stage" in /etc/apache2/conf.d:

User dw Group dw UseCanonicalName off StartServers 1 MaxSpareServers 2 MinSpareServers 1 DocumentRoot /home/dw/htdocs PerlSetEnv LJHOME /home/dw/dw PerlPassEnv LJHOME PerlRequire /home/dw/dw/cgi-bin/modperl.pl

Then disable the default site:

a2dissite default

You might also have to enable the Perl Apache Request module:

a2enmod apreq

If you don't want this warning:

[Thu Jan 15 01:46:54 2009] [warn] NameVirtualHost *:80 has no VirtualHosts ... waiting [Thu Jan 15 01:46:55 2009] [warn] NameVirtualHost *:80 has no VirtualHosts ...done.

Then use  and put a # in front of the NameVirtualHost line:


 * 1) NameVirtualHost *:80

Restart the server:

/etc/init.d/apache2 restart

Have a look at your new DW instance
Congratulations! You now have a working (though minimal) Dreamwidth install. If you point your web browser at your server, you should see a bare-looking welcome page.

Further setup
There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:


 * TheSchwartz Setup - TheSchwartz is needed for a handful of features - notably comment posting
 * Subdomain setup
 * Statistics setup
 * Generating documentation
 * Allow users to register
 * Setting up Search

There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.

Starting development
If you haven't already, see Dev Getting Started.