Difference between revisions of "Dreamhack Getting Started"

From Dreamwidth Notes
Jump to: navigation, search
(Step-by-step guide to setting up your Dreamhack account)
 
(Edit the config files)
 
(55 intermediate revisions by 14 users not shown)
Line 1: Line 1:
[[Category:Development]]
 
[[Cagtegory:Dreamhack]]
 
[[Category:Getting Started]]
 
 
 
The [http://hack.dreamwidth.net Dreamhack] service is a hosted development sandbox environment where you can do Dreamwidth development without having to have your own server. Dreamhack accounts are available free of charge for anyone who's interested in working on the Dreamwidth code.
 
The [http://hack.dreamwidth.net Dreamhack] service is a hosted development sandbox environment where you can do Dreamwidth development without having to have your own server. Dreamhack accounts are available free of charge for anyone who's interested in working on the Dreamwidth code.
  
 
When you get your Dreamhack account first created, most of the setup will be done for you. There are still a bunch of things you should do at first to make sure that your development environment is properly set up for you to work on, though. This guide is a walkthrough of all the steps you should take (in order) when you first log into your Dreamhack account.  
 
When you get your Dreamhack account first created, most of the setup will be done for you. There are still a bunch of things you should do at first to make sure that your development environment is properly set up for you to work on, though. This guide is a walkthrough of all the steps you should take (in order) when you first log into your Dreamhack account.  
  
You should also read [[Beginning dev checklist]] for more background information.
+
You should also read [[Beginning dev checklist]] for more background information; if you are on Windows, [[Newbie Guide: Getting Started on Windows]] might be useful. If you wind up in difficulty, take a look at [[Dreamhack Troubleshooting]].
  
==Get a SSH client==
+
== Basic setup ==
  
To connect to your Dreamhack and work on things, you'll need a program that will do SSH, or "secure shell". It lets you connect to the Dreamhack server and work with the files, which you can't do through your browser. If you're running Windows, download [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal on the Mac) will let you connect.
+
=== Get a SSH client ===
  
==Connect to your Dreamhack==
+
To connect to your Dreamhack and work on things, you'll need a program that will do SSH, or "secure shell". It lets you connect to the Dreamhack server and work with the files, which you can't do through your browser. If you're running Windows, download [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal on the Mac) will let you connect. If you're on a Mac, you can also [[Connecting_to_the_Dreamhack_server_with_Cyberduck | use Cyberduck]].
  
Once you have an SSH client, you can connect to Dreamhack. In PuTTY, put "hack.dreamwidth.net" for the server, and use the username and password you were given in your welcome email. In Terminal, type:
+
=== Connect to your Dreamhack ===
  
<source lang="bash">ssh dh-username@hack.dreamwidth.net</source>
+
Once you have an SSH client, you can connect to Dreamhack. In PuTTY, put "ssh-hack.dreamwidth.net" for the server, and use the username and password you were given in your welcome email. In Terminal, type:
  
==Change your password==
+
<source lang="bash">ssh dh-username@ssh-hack.dreamwidth.net</source>
 +
 
 +
If this is the first time you've connected to the server, your SSH client will inform you that it hasn't seen this host before, and ask you to verify the fingerprint of the host key. For '''ssh-hack.dreamwidth.net''', this fingerprint will show one of the following, depending on your SSH client:
 +
 
 +
* <tt>SHA256:TAtO07wp/zu1L4lIoYPnJh9pficQUcBsl9aqZiEM6UE</tt>
 +
* <tt>SHA256:jU9iFXflDW/FMRAFDaR0HMZ3zwsNxBa333A4Pl7ys1w</tt>
 +
* <tt>73:10:37:bc:47:4b:24:dc:50:5f:7e:af:f7:b4:a0:b6</tt>
 +
* <tt>29:18:e9:63:c5:b5:79:5c:bd:d6:3b:63:ce:7a:83:85</tt>
 +
 
 +
Compare the fingerprint to what you see above, and if one of the above fingerprints match, click (or type) "Yes" to connect. If you do ''not'' see the fingerprint listed above, please contact Sophira on [[IRC]] or by email.
 +
 
 +
=== Change your password ===
  
 
Change your password to something you can remember. Type:
 
Change your password to something you can remember. Type:
Line 27: Line 34:
 
and follow the prompts.
 
and follow the prompts.
  
==Set the system password==
+
=== Set the system password ===
  
 
Your Dreamhack installation has a user named 'system', which has all the admin privs and owns all the system styles. The system account is the first account on the server. You need to set a password so you can log into it. Type:
 
Your Dreamhack installation has a user named 'system', which has all the admin privs and owns all the system styles. The system account is the first account on the server. You need to set a password so you can log into it. Type:
Line 35: Line 42:
 
Pick something you'll remember. (If you ever forget the system password, though, you can just run make_system.pl again and re-set it.)
 
Pick something you'll remember. (If you ever forget the system password, though, you can just run make_system.pl again and re-set it.)
  
==Edit the config files==
+
=== Get acquainted with the command line ===
 +
 
 +
If you haven't used a command line before, it can be a strange experience. You may want to check out the [[Command line tutorial]] before carrying on, as it explains a few of the concepts that we'll be using in the sections below.
 +
 
 +
=== Edit the config files ===
  
 
You'll need to edit a few config files to make for easier development. Do the following:
 
You'll need to edit a few config files to make for easier development. Do the following:
  
 
<source lang="bash">cd $LJHOME
 
<source lang="bash">cd $LJHOME
nano -w etc/config.pl</source>
+
nano ext/local/etc/config.pl</source>
  
(Or whatever editor you prefer.)  
+
(The "nano" text editor is probably the best one for new developers. If you're more experienced, or want to try something a little more powerful, check out the list of editors at the [[Configuring editors]] page.)
  
Values you may want to change and/or set:
+
You'll need to use the arrow keys to scroll through the file when using nano, because it won't respond to the mouse.
 +
 
 +
Take a look for the following line:
 +
 
 +
<source lang="bash">$USE_ACCT_CODES = 1;</source>
 +
 
 +
This line governs whether invite codes are used. If the value is 1, then new accounts will need invites before they can be created. If the value is 0, then anybody can create an account. You can edit this value, if you like.
  
<source lang="bash">$USE_ACCT_CODES = 1;
+
When you're done, press Ctrl-X to exit nano. It'll ask you if you want to save - press Y. It'll then ask you where you want to save the file. At this point, just press RETURN; you already told nano where to save it when you launched it, and pressing RETURN tells nano just to use that. You'll then find yourself back at the command line.
# This governs invite codes. 1 = new accounts need invites; 0 = anyone can create an account.</source>
+
  
You should also edit your config-local.pl for some other local settings:
+
You should also edit your <tt>$LJHOME/ext/local/etc/config-local.pl</tt> file for some other local settings:
  
 
<source lang="bash">cd $LJHOME
 
<source lang="bash">cd $LJHOME
nano -w etc/config-local.pl</source>
+
nano -w ext/local/etc/config-local.pl</source>
  
 
Values you may want to change and/or set:
 
Values you may want to change and/or set:
Line 59: Line 75:
 
$SITENAMESHORT = "YourSite";
 
$SITENAMESHORT = "YourSite";
 
$SITENAMEABBREV = "YS";
 
$SITENAMEABBREV = "YS";
# these are all used in various places throughout the site
+
# these are all used in various places throughout the site</source>
  
$MAIL_TO_THESCHWARTZ = 1;
+
Remember to save and exit when you're done.
# change this to 0 and/or comment it out to make your hack send email
+
# without having to have TheSchwartz running</source>
+
  
==Set up TheSchwartz==
+
=== Start Apache ===
  
[[TheSchwartz]] is a worker manager system that runs things like the importer, the private messaging function, the crossposter, and all of the event notifications. If you plan on working with any of those, you'll need to set up TheSchwartz on your Dreamhack.
+
Once all of this is done, you're ready to start Apache. Type:
  
The page for [[TheSchwartz Setup]]  assumes that you're running your own installation, not a Dreamhack. The instructions for Dreamhacks are a little diffent. Edit your <tt>etc/config-local.pl</tt> and find the section marked @THESCHWARTZ_DBS. Edit it to look like this:
+
<source lang="bash">start-apache</source>
  
<source lang="bash">@THESCHWARTZ_DBS = (
+
Your Dreamhack will now appear at http://www.yourusername.hack.dreamwidth.net.
        {
+
            dsn => 'dbi:mysql:dreamhack_yourusername;host=localhost',
+
            user => 'dh_yourusername',
+
            pass => $DW::PRIVATE::THESCHWARTZ_DBS{pass},
+
            },
+
        );</source>
+
  
(Obviously, replace "yourusername" with your Dreamhack username. For instance, if your username is dh_rahaeli, you should use "dh_rahaeli" as the username.)
+
=== Hack away! ===
  
Then, edit your <tt>etc/config-private</tt> and find the section marked %THESCHWARTZ_DBS. Edit it to look like this:
+
You can now start hacking! For more information, check out the [[:Category: Development]] category, especially the [[Beginning dev checklist]], [[Dev Getting Started]], and [[Programming Guidelines]] pages. If you run into problems, the [irc://irc.freenode.net/dreamwidth-dev #dreamwidth-dev] [[IRC]] channel and the <dwcomm>dw_dev_training</dwcomm> community are good places to go for help.
  
<source lang="bash">%THESCHWARTZ_DBS = (
+
A quick and easy way to populate your Dreamhack with lots of users is to run Sophie's [[Script:_create-users]].
        pass => 'password',
+
);</source>
+
  
Your password should be your database password, NOT the password you use to connect to the Dreamhack machine.
+
We're looking forward to seeing your first patch!
  
==Back up your configs==
+
=== Stop Apache ===
  
For safety and paranoia's sake, it's a good idea to back up all three of your config files in case a change accidentally overwrites them:
+
At the end of a Dreamhack session, remember to <code>stop-apache</code> so that the Dreamhack machine isn't running too many web servers that aren't being used.
  
<source lang="bash">cp $LJHOME/etc/config-local.pl $LJHOME/etc/config-local.pl.bak
+
== Extended Setup ==
cp $LJHOME/etc/config-private.pl $LJHOME/etc/config-private.pl.bak
+
cp $LJHOME/etc/config.pl $LJHOME/etc/config.pl.bak</source>
+
  
==Protect your configs from updates==
+
=== Configuring BlobStore ===
  
Code updates can wind up wiping out your config files, which will cause you to get errors connecting to your database. To prevent this from happening, set all of your config files to be locally tracked:
+
If you want to upload user icons, or if you see any errors of the form <b><code>Must configure @LJ::BLOBSTORES</code></b> in your logs, then you will need to configure [[BlobStore]] on your Dreamhack.
  
<source lang="bash">cd $LJHOME
+
Dreamwidth uses S3 in production for data storage, but a development environment can use local disk storage instead. To set it up, edit your <code>~/dw/ext/local/etc/config-private.pl</code> file and uncomment these lines:
mkdir cvs/local cvs/local/etc cvs/local/cvs
+
cp etc/config-local.pl cvs/local/etc
+
cp etc/config.pl cvs/local/etc
+
cp etc/config-private.pl cvs/local/etc</source>
+
  
Create <tt>$LJHOME/cvs/local/cvs/multicvs-private.conf</tt> and put this in it:
+
<source lang=perl>
 +
    @BLOBSTORES = (
 +
        # Local disk configuration, can be used to store everything on one machine
 +
        localdisk => {
 +
            path => "$LJ::HOME/var/blobstore",
 +
        },
 +
    );
 +
</source>
  
local .
+
Save the file and restart apache for these changes to take effect.
  
Then, copy it to the main <tt>cvs</tt> directory:
+
== Tips and Advice ==
  
cp $LJHOME/cvs/local/cvs/multicvs-private.conf $LJHOME/cvs/
+
=== Update your code regularly ===
  
From now on, if you want to make changes to any of your configs, do them to the versions in <tt>cvs/local/etc/</tt>. (You'll need to do this if any new features come about that involve config changes; it's a good idea to compare your local versions with the versions in Mercurial from time to time.)
+
Before you do any development work, it's a good idea to update your code to the most recent version. See [[Dev Maintenance]] for instructions.
  
==Start Apache==
+
=== Get familiar with version control ===
  
Once all of this is done, you're ready to start Apache. Type:
+
Browse through [[Git How To]] and get familiar with how things work -- bookmark it so you can refer back to each section as needed.
  
<source lang="bash">start-apache</source>
+
=== A note about workers ===
  
Your Dreamhack will now appear at http://yourusername.hack.dreamwidth.net.
+
Some features run asynchronously. That means they don't happen immediately when they are fired off. Instead, they get queued up to run later via [[Workers]], at pre-programmed periods of time.
  
==Update your code regularly==
+
Features that run asynchronously via workers are either:
 +
* process heavy, such as search and notifications, or
 +
* require communication with remote sites we don't control, such as crossposts and imports.
  
Before you do any development work, it's a good idea to update your code to the most recent version. See [[Dev Maintenance]] for instructions.
+
If you aren't working on these features, then you don't need to worry about workers.
  
==Get familiar with version control==
+
If you are working on these features, there are two different kinds of workers: [[TheSchwartz]] and [[Gearman]]. If you need them, recent Dreamhacks are already set up to let you use TheSchwartz (and Gearman?). If your Dreamhack is very old you can set up the TheSchwartz database using:
  
Read [[Version control]] and get familiar with how things work. You don't need to use Mercurial queues to manage your patches -- it's perfectly possible to just use $LJHOME/bin/cvsreport.pl every time you want to make a patch.  
+
<source lang="bash">$LJHOME/bin/ljdb < /usr/share/doc/libtheschwartz-perl/schema.sql</source>
  
==Hack away!==
+
then restarting Apache.
  
You can now start hacking! For more information, check out the [[:Category: Development]] category, especially the [[Beginning dev checklist]], [[Dev getting started]], and [[Programming guidelines]] pages. If you run into problems, the [irc://irc.dreamwidth.org/dw_kindergarten #dw_kindergarten] [[IRC]] channel and the <dwcomm>dw_dev_training</dwcomm> community are good places to go for help.
+
[[Category:Development]]
 
+
[[Category:Dreamhack]]
We're looking forward to seeing your first patch!
+
[[Category:Getting Started]]

Latest revision as of 21:25, 31 October 2023

The Dreamhack service is a hosted development sandbox environment where you can do Dreamwidth development without having to have your own server. Dreamhack accounts are available free of charge for anyone who's interested in working on the Dreamwidth code.

When you get your Dreamhack account first created, most of the setup will be done for you. There are still a bunch of things you should do at first to make sure that your development environment is properly set up for you to work on, though. This guide is a walkthrough of all the steps you should take (in order) when you first log into your Dreamhack account.

You should also read Beginning dev checklist for more background information; if you are on Windows, Newbie Guide: Getting Started on Windows might be useful. If you wind up in difficulty, take a look at Dreamhack Troubleshooting.

Basic setup

Get a SSH client

To connect to your Dreamhack and work on things, you'll need a program that will do SSH, or "secure shell". It lets you connect to the Dreamhack server and work with the files, which you can't do through your browser. If you're running Windows, download PuTTY. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal on the Mac) will let you connect. If you're on a Mac, you can also use Cyberduck.

Connect to your Dreamhack

Once you have an SSH client, you can connect to Dreamhack. In PuTTY, put "ssh-hack.dreamwidth.net" for the server, and use the username and password you were given in your welcome email. In Terminal, type:

ssh dh-username@ssh-hack.dreamwidth.net

If this is the first time you've connected to the server, your SSH client will inform you that it hasn't seen this host before, and ask you to verify the fingerprint of the host key. For ssh-hack.dreamwidth.net, this fingerprint will show one of the following, depending on your SSH client:

  • SHA256:TAtO07wp/zu1L4lIoYPnJh9pficQUcBsl9aqZiEM6UE
  • SHA256:jU9iFXflDW/FMRAFDaR0HMZ3zwsNxBa333A4Pl7ys1w
  • 73:10:37:bc:47:4b:24:dc:50:5f:7e:af:f7:b4:a0:b6
  • 29:18:e9:63:c5:b5:79:5c:bd:d6:3b:63:ce:7a:83:85

Compare the fingerprint to what you see above, and if one of the above fingerprints match, click (or type) "Yes" to connect. If you do not see the fingerprint listed above, please contact Sophira on IRC or by email.

Change your password

Change your password to something you can remember. Type:

passwd

and follow the prompts.

Set the system password

Your Dreamhack installation has a user named 'system', which has all the admin privs and owns all the system styles. The system account is the first account on the server. You need to set a password so you can log into it. Type:

$LJHOME/bin/upgrading/make_system.pl

Pick something you'll remember. (If you ever forget the system password, though, you can just run make_system.pl again and re-set it.)

Get acquainted with the command line

If you haven't used a command line before, it can be a strange experience. You may want to check out the Command line tutorial before carrying on, as it explains a few of the concepts that we'll be using in the sections below.

Edit the config files

You'll need to edit a few config files to make for easier development. Do the following:

cd $LJHOME
nano ext/local/etc/config.pl

(The "nano" text editor is probably the best one for new developers. If you're more experienced, or want to try something a little more powerful, check out the list of editors at the Configuring editors page.)

You'll need to use the arrow keys to scroll through the file when using nano, because it won't respond to the mouse.

Take a look for the following line:

$USE_ACCT_CODES = 1;

This line governs whether invite codes are used. If the value is 1, then new accounts will need invites before they can be created. If the value is 0, then anybody can create an account. You can edit this value, if you like.

When you're done, press Ctrl-X to exit nano. It'll ask you if you want to save - press Y. It'll then ask you where you want to save the file. At this point, just press RETURN; you already told nano where to save it when you launched it, and pressing RETURN tells nano just to use that. You'll then find yourself back at the command line.

You should also edit your $LJHOME/ext/local/etc/config-local.pl file for some other local settings:

cd $LJHOME
nano -w ext/local/etc/config-local.pl

Values you may want to change and/or set:

$SITENAME = "Some LiveJournal Site";
$SITENAMESHORT = "YourSite";
$SITENAMEABBREV = "YS";
# these are all used in various places throughout the site

Remember to save and exit when you're done.

Start Apache

Once all of this is done, you're ready to start Apache. Type:

start-apache

Your Dreamhack will now appear at http://www.yourusername.hack.dreamwidth.net.

Hack away!

You can now start hacking! For more information, check out the Category: Development category, especially the Beginning dev checklist, Dev Getting Started, and Programming Guidelines pages. If you run into problems, the #dreamwidth-dev IRC channel and the [info]dw_dev_training community are good places to go for help.

A quick and easy way to populate your Dreamhack with lots of users is to run Sophie's Script:_create-users.

We're looking forward to seeing your first patch!

Stop Apache

At the end of a Dreamhack session, remember to stop-apache so that the Dreamhack machine isn't running too many web servers that aren't being used.

Extended Setup

Configuring BlobStore

If you want to upload user icons, or if you see any errors of the form Must configure @LJ::BLOBSTORES in your logs, then you will need to configure BlobStore on your Dreamhack.

Dreamwidth uses S3 in production for data storage, but a development environment can use local disk storage instead. To set it up, edit your ~/dw/ext/local/etc/config-private.pl file and uncomment these lines:

     @BLOBSTORES = (
         # Local disk configuration, can be used to store everything on one machine
         localdisk => {
             path => "$LJ::HOME/var/blobstore",
         },
     );

Save the file and restart apache for these changes to take effect.

Tips and Advice

Update your code regularly

Before you do any development work, it's a good idea to update your code to the most recent version. See Dev Maintenance for instructions.

Get familiar with version control

Browse through Git How To and get familiar with how things work -- bookmark it so you can refer back to each section as needed.

A note about workers

Some features run asynchronously. That means they don't happen immediately when they are fired off. Instead, they get queued up to run later via Workers, at pre-programmed periods of time.

Features that run asynchronously via workers are either:

  • process heavy, such as search and notifications, or
  • require communication with remote sites we don't control, such as crossposts and imports.

If you aren't working on these features, then you don't need to worry about workers.

If you are working on these features, there are two different kinds of workers: TheSchwartz and Gearman. If you need them, recent Dreamhacks are already set up to let you use TheSchwartz (and Gearman?). If your Dreamhack is very old you can set up the TheSchwartz database using:

$LJHOME/bin/ljdb < /usr/share/doc/libtheschwartz-perl/schema.sql

then restarting Apache.