Difference between revisions of "Beginning dev checklist"

From Dreamwidth Notes
Jump to: navigation, search
(Making your changes into a patch)
m (Updating SSH hostname for Dreamhacks.)
 
(37 intermediate revisions by 12 users not shown)
Line 1: Line 1:
{{Expand|text=See inline expansion notes.}}
+
{{Cleanup|text=Most of this should go into bullet points in [[Dev Getting Started]] or [[Dreamhack getting started]], and then this page can be blanked and redirected.}}
  
We believe that our mentoring process is an investment in our future and in the future of Open Source. By teaching people -- no matter how experienced or inexperienced -- how to contribute to our codebase, we're building our own next generation of contributors. Every hour that we spend training someone to contribute is paid back tenfold, in multiple ways: we get more patches submitted, we get more people who can mentor other people, we get better documentation, and we get more people into the Open Source world.
 
  
If you're interested in beginning development with us, we'll teach you. Here's the absolute basic framework of what you need to know, what you need to have, and what you need to learn. We'll fill it in with links to Wiki articles explaining each step of the process.
+
== Working with a development environment ==
  
Be aware that a lot of this article comes from the viewpoint that you'll be using a [[Dreamhacks|Dreamhack]].
 
 
== Setting up and working with a development environment ==
 
 
=== Getting a development environment ===
 
 
The absolute first thing you'll need is a development environment: someplace where you can host the code you're going to be working on. This will be a complete local installation of the Dreamwidth service, running on a separate server. If you're comfortable with installing the code yourself, see [[Dev Getting Started]]. If you absolutely don't know where to start, are overwhelmed at the idea of trying, we offer hosted development installations on our [http://hack.dreamwidth.net Dreamhack] service: we'll do the initial install for you. (You'll have to keep it upgraded after that.)
 
  
 
=== Working with Unix ===
 
=== Working with Unix ===
Line 19: Line 11:
 
=== The location of $LJHOME ===
 
=== The location of $LJHOME ===
  
Some instructions on the wiki will mention $LJHOME (must be all caps). This is where your Dreamwidth code can be found. You can use it as-is.
+
Some instructions on the wiki will mention $LJHOME (must be all caps). This is where your Dreamwidth code can be found. You can type it as-is; there's no need to replace it with anything else.
  
For example, on a Dreamhack, <code>cd $LJHOME/...</code> is the same as <code>cd ~/dw/...</code> (as this is the directory that $LJHOME points to), and will change your directory to ~/dw/...
+
On a Dreamhack, $LJHOME points to the <code>~/dw</code> directory. Thus, <code>cd $LJHOME/etc</code> (for example) is the same as <code>cd ~/dw/etc</code>, and will change your directory to ~/dw/etc.
  
 
=== Accessing your files ===
 
=== Accessing your files ===
  
Once you have a Dreamhack, you'll need to access the files on them (in order to change them). This involves downloading a program that will do SSH, or "secure shell". It lets you connect to the other site (in this case your Dreamhack) and work with the files, which you can't do through your browser. If you're running Windows, download [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal) will let you connect.
+
Once you have a development environment, you'll need to access the files on them (in order to change them). This involves downloading a program that will do SSH, or "secure shell". It lets you connect to the other site (in this case your dev server) and work with the files, which you can't do through your browser. If you're running Windows, download [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal on the Mac) will let you connect.
  
=== Connecting to your development environment ===
+
Use your SSH system to go to <code>ssh-hack.dreamwidth.net</code>. Enter the username and password provided in your Dreamhack setup email.
  
Once you have an SSH client, you can connect to Dreamhack. In PuTTY, put "hack.dreamwidth.net" for the server, and use the username and password you were given in your welcome email. In Terminal, type:
+
===Set the system user password===
  
<source lang="bash">ssh dh-username@hack.dreamwidth.net</source>
+
Your Dreamhack installation has a user named 'system', which has all the admin privs and owns all the system styles. The system account is the first account on the server. You need to set a password so you can log into it. Type:
  
The first time you log in, you'll probably want to change your password. Type:
+
<source lang="bash">$LJHOME/bin/upgrading/make_system.pl</source>
  
<source lang="bash">passwd</source>
+
Pick something you'll remember. (If you ever forget the system password, though, you can just run make_system.pl again and re-set it.)
  
You'll also need to set the LJ system password. Type:
+
===Set up for test users===
  
<source lang="bash">$LJHOME/bin/upgrading/make_system.pl</source>
+
You are probably going to want to create test users on your Dreamhack. By default your Dreamhack installation will ask for invite codes. You can generate your own invite codes on your Dreamhack, or disable this. Instructions on disabling this are [[#Allow users to register without invite codes | included further down]].  
  
When you're ready, start Apache. Type:
+
You are likely to want at least two test accounts, for example so one can subscribe to inbox/email notifications of entries posted by another. You can create test accounts as you find you need them.
 
+
<source lang="bash">start-apache</source>
+
  
 
== Start programming ==
 
== Start programming ==
Line 49: Line 39:
 
=== Editors ===
 
=== Editors ===
  
Once you've done all that, you're ready to start programming! You're going to need an editor: something that will let you edit the files you're working with. There are two ways you can do this. You can either download the files from Dreamhack to your local machine, edit them, and then re-upload them, or you can edit them directly on Dreamhack itself. If you want to edit them on Dreamhack itself, you're going to need to learn one of the Unix editors. The question of <em>which</em> editor you should be using is going to be personal preference, and the "editor war" is a hotly-debated topic. For an absolute beginner, the best editor to use is probably nano, an open-source version of the "pico" editor. it doesn't have everything, but it is very, <em>very</em> quickly learnable. (Other editors in the server include emacs, vim, and joe, but nano is the easiest. "pico" isn't installed on the server; if you try to use it, it'll run nano instead.)
+
Once you've done all that, you're ready to start programming! You're going to need to use an editor: something that will let you edit the files you're working with. There are two ways you can do this:
 +
 
 +
* You can download the files from your Dreamhack to your local machine, edit them, and then re-upload them.
 +
* You can edit them directly on the Dreamhack itself.
 +
 
 +
If you want to edit them on the Dreamhack, you're going to need to learn one of the several Unix editors that are installed on the system. The question of <em>which</em> editor you should be using is going to be personal preference, and the "editor war" is a hotly-debated topic.
 +
 
 +
For an absolute beginner, the best editor to use is probably <code>nano</code>, an open-source version of the "pico" editor. it doesn't have everything, but it is very, <em>very</em> quickly learnable. (Other editors in the server include <code>emacs</code>, <code>vim</code>, and <code>joe</code>, but <code>nano</code> is the easiest. "pico" isn't installed on the server; if you try to use it, it'll run nano instead.)
  
 
=== Working with nano ===
 
=== Working with nano ===
  
One thing you absolutely do need to keep in mind, though, is that by default nano quietly adds linebreaks to the file that you're working on, not just word-wraps it for display in the editor. This can cause trouble with generating patches, and because it's indistinguishable from word-wrapping in the editor, it can take you a while to diagnose it. To prevent this from happening, when you call nano, do it with the -w flag, which will turn off auto-linebreaking:
+
One thing you absolutely do need to keep in mind, though, is that by default nano will word-wrap the file that you're working on, and when it does this, it quietly adds *real* linebreaks to the file. This can cause trouble with generating patches, and because it looks like it's just a display word-wrap, rather than an actual word-wrap, it can take you a while to diagnose it. To prevent this from happening, when you call nano, do it with the -w flag, which will turn off auto-linebreaking for that session:
  
 
<source lang="bash">nano -w filename</source>
 
<source lang="bash">nano -w filename</source>
  
You may want to make this a default by aliasing the command:
+
You may want to make this a default by editing the nano config file:
  
<source lang="bash">echo 'alias nano="nano -w"' >> ~/.bashrc</source>
+
<source lang="bash">echo 'set nowrap' >> ~/.nanorc</source>
 
+
Now, whenever you run 'nano', it'll run 'nano -w' instead.
+
  
 
=== Allow users to register without invite codes ===
 
=== Allow users to register without invite codes ===
Line 72: Line 67:
 
nano -w etc/config.pl</source>
 
nano -w etc/config.pl</source>
  
This will open the config file in nano. Have a skim through, as there are lots of useful options here. To turn invite codes off, scroll down to $USE_ACCT_CODES = 1 and change it to 0. To save this, use Ctrl-O and then Enter, and exit using Ctrl-X. You can then go to http://www.yourname.hack.dreamwidth.net and start playing.
+
This will open the config file in nano. Have a skim through, as there are lots of useful options here. To turn invite codes off, scroll down to $USE_ACCT_CODES = 1 and change it to 0. To save this, use Ctrl-O and then Enter, and exit using Ctrl-X.  
  
( ... insert how-to-edit-a-file here )
+
Once you've done that, restart your hack by typing "stop-apache", then "start-apache". (Every time you change a file that ends in .pm or .pl, it's a good idea to restart Apache again to make sure the modules reload.)
 +
 
 +
You can then go to http://www.yourname.hack.dreamwidth.net/ (replacing 'yourname' with the name you chose when applying for a Dreamhack) and start playing. The front page of your hack will now give you a message about being able to join without an invite code for a week, and offer a link to do so. You can ignore the "This week only" part - that's just the text that was initially used on the regular site when we first turned off invite codes.
 +
 
 +
=== Or, create a promo code ===
 +
 
 +
If you don't want to turn off invite codes entirely, to prevent spam account registration, you can make a "promo code" that will let you create users whenever you want, without having to generate invite codes directly.
 +
 
 +
First, go to http://www.yourname.hack.dreamwidth.net/admin/priv (while logged in as the system user or another user with admin:*) and grant yourself the 'payments' priv. Then, go to http://www.yourname.hack.dreamwidth.net/admin/invites/promo. Select "Create New", check the "Active" box, and input a code (letters/numbers only; case-sensitive) and a number of accounts the code will work to create. Once you hit save, the promo code will be activated and will work the same way as an invite code will.
  
 
=== Layout of the code directory ===
 
=== Layout of the code directory ===
  
Your Dreamwidth code directory is set up with two separate copies of the code: one that actually runs the site (in ~/dw/*) and one that is in your 'cvs' directory (~/dw/cvs/dw-free or ~/dw/cvs/dw-nonfree), which is what you use to sync back and forth with the 'official' copy of the code. (Think of the whole thing like an endless series of comparisons: you're comparing the version you're working on with the version you have set aside, which you <em>then</em> compare against the master version that's stored on our code repository.)
+
All of the code is contained in the $LJHOME directory.  On Dreamhacks, this is in: <tt>~/dw</tt>. When you make changes to the code in this directory and restart the server, your changes will show up.
  
Which version you edit is up to you. Everyone does it a bit differently. You can either edit in the live directory structure (~/dw/*) and then copy over to the cvs directory (~/dw/cvs/dw-free), or edit in the cvs directory and then copy over to the live directory. Either way, you're going to need to have your changes in both places: you need to have them in the live directory so you can test your change in your browser, and you need to have them in the cvs directory so you can generate a patchfile (which is a file that anyone else can apply against the code to make the same changes you're making).
+
You can see what changes you have made by doing <code>git diff</code>: the output shows all local changes to the branch you're in since the last commit.
  
( ... insert more background information on version control, patch files, diffs, copying over, and provide step-by-step list of what each workflow consists of, its benefits, and its drawbacks )
+
You can revert an the original committed version of the code by [INSERT GIT INSTRUCTIONS]
  
 
=== Finding what you're looking for in the code ===
 
=== Finding what you're looking for in the code ===
Line 92: Line 95:
 
( ...insert more about how to recover from a disaster, aka What Happens When You Break It )
 
( ...insert more about how to recover from a disaster, aka What Happens When You Break It )
  
== Submitting your changes to Dreamwidth ==
+
=== Getting back to a clean slate ===
 +
Sometimes when you're investigating an issue, you'll end up with a bunch of debug code hanging around in random files. To see all your changes, run:
 +
 
 +
    git status
 +
 
  
=== Making your changes into a patch ===
+
To revert all of your changes and get back to a clean slate, run:
  
Once you've made your changes, and verified that they're working the way you want them to &mdash; testing every single usage case you think could possibly be affected by what you've changed &mdash; you need to submit your change to [[Bugzilla]], so it can be applied to the official code repository. You <em>don't</em> send the whole file; you're going to generate what's known as a "patch" file, which highlights the bits that you've changed and only the bits that you've changed. One of the people (known as a 'committer') who maintain the official code repository will apply that patch to their own development environment, verify that it works, and if it works, commit it to the main code repository.
+
    git checkout .
  
You don't have to manually write the patch file. There are tools that will create them for you. Again, everyone's workflow is slightly different, and there are a few different ways to generate patch files.
+
To revert only the changes in specific files or directories, run:
  
See [[Dev Patches]] and [[Dev version control]] for how to create a patch file, and [[Dev Committing Guidelines]] also has some useful information. You'll probably also need to read [[Dev maintenance]] if you haven't already.
+
    git checkout dir/subdir dir/file.txt
  
=== Uploading your patch to Bugzilla ===
 
  
Once you've got your patch file, you will upload it to Bugzilla. See [[Bugzilla workflow]] for the basics of how to get your patch file into Bugzilla and how you need to handle it.
 
  
* And that's it!
+
[[Category: Development]]

Latest revision as of 04:44, 4 April 2017

Cleanup: Most of this should go into bullet points in Dev Getting Started or Dreamhack getting started, and then this page can be blanked and redirected. Please refer to the Wiki Style Guide and Manual of Style pages.


Working with a development environment

Working with Unix

The Dreamwidth code runs under Unix (specifically, Ubuntu Linux). If you've never used Unix before, you might want to read over a tutorial on how to approach the system, how to use the command line, how to create files, delete files, move files, and other useful things. For the rest of this list, I'll assume you're familiar with the basics of directory structures. In particular, you use the symbol ~ to represent your home directory: the top-level directory that everything else goes in.

The location of $LJHOME

Some instructions on the wiki will mention $LJHOME (must be all caps). This is where your Dreamwidth code can be found. You can type it as-is; there's no need to replace it with anything else.

On a Dreamhack, $LJHOME points to the ~/dw directory. Thus, cd $LJHOME/etc (for example) is the same as cd ~/dw/etc, and will change your directory to ~/dw/etc.

Accessing your files

Once you have a development environment, you'll need to access the files on them (in order to change them). This involves downloading a program that will do SSH, or "secure shell". It lets you connect to the other site (in this case your dev server) and work with the files, which you can't do through your browser. If you're running Windows, download PuTTY. If you're on a Mac or a Linux distribution, you won't need a separate program; the Terminal (Applications/Utilities/Terminal on the Mac) will let you connect.

Use your SSH system to go to ssh-hack.dreamwidth.net. Enter the username and password provided in your Dreamhack setup email.

Set the system user password

Your Dreamhack installation has a user named 'system', which has all the admin privs and owns all the system styles. The system account is the first account on the server. You need to set a password so you can log into it. Type:

$LJHOME/bin/upgrading/make_system.pl

Pick something you'll remember. (If you ever forget the system password, though, you can just run make_system.pl again and re-set it.)

Set up for test users

You are probably going to want to create test users on your Dreamhack. By default your Dreamhack installation will ask for invite codes. You can generate your own invite codes on your Dreamhack, or disable this. Instructions on disabling this are included further down.

You are likely to want at least two test accounts, for example so one can subscribe to inbox/email notifications of entries posted by another. You can create test accounts as you find you need them.

Start programming

Editors

Once you've done all that, you're ready to start programming! You're going to need to use an editor: something that will let you edit the files you're working with. There are two ways you can do this:

  • You can download the files from your Dreamhack to your local machine, edit them, and then re-upload them.
  • You can edit them directly on the Dreamhack itself.

If you want to edit them on the Dreamhack, you're going to need to learn one of the several Unix editors that are installed on the system. The question of which editor you should be using is going to be personal preference, and the "editor war" is a hotly-debated topic.

For an absolute beginner, the best editor to use is probably nano, an open-source version of the "pico" editor. it doesn't have everything, but it is very, very quickly learnable. (Other editors in the server include emacs, vim, and joe, but nano is the easiest. "pico" isn't installed on the server; if you try to use it, it'll run nano instead.)

Working with nano

One thing you absolutely do need to keep in mind, though, is that by default nano will word-wrap the file that you're working on, and when it does this, it quietly adds *real* linebreaks to the file. This can cause trouble with generating patches, and because it looks like it's just a display word-wrap, rather than an actual word-wrap, it can take you a while to diagnose it. To prevent this from happening, when you call nano, do it with the -w flag, which will turn off auto-linebreaking for that session:

nano -w filename

You may want to make this a default by editing the nano config file:

echo 'set nowrap' >> ~/.nanorc

Allow users to register without invite codes

Note: Maybe should reorganize this so this is closer up to the first setups?

A useful first edit is to turn invite codes off in etc/config.pl, to make it quicker to set up dummy accounts for playing around. Type:

cd $LJHOME
nano -w etc/config.pl

This will open the config file in nano. Have a skim through, as there are lots of useful options here. To turn invite codes off, scroll down to $USE_ACCT_CODES = 1 and change it to 0. To save this, use Ctrl-O and then Enter, and exit using Ctrl-X.

Once you've done that, restart your hack by typing "stop-apache", then "start-apache". (Every time you change a file that ends in .pm or .pl, it's a good idea to restart Apache again to make sure the modules reload.)

You can then go to http://www.yourname.hack.dreamwidth.net/ (replacing 'yourname' with the name you chose when applying for a Dreamhack) and start playing. The front page of your hack will now give you a message about being able to join without an invite code for a week, and offer a link to do so. You can ignore the "This week only" part - that's just the text that was initially used on the regular site when we first turned off invite codes.

Or, create a promo code

If you don't want to turn off invite codes entirely, to prevent spam account registration, you can make a "promo code" that will let you create users whenever you want, without having to generate invite codes directly.

First, go to http://www.yourname.hack.dreamwidth.net/admin/priv (while logged in as the system user or another user with admin:*) and grant yourself the 'payments' priv. Then, go to http://www.yourname.hack.dreamwidth.net/admin/invites/promo. Select "Create New", check the "Active" box, and input a code (letters/numbers only; case-sensitive) and a number of accounts the code will work to create. Once you hit save, the promo code will be activated and will work the same way as an invite code will.

Layout of the code directory

All of the code is contained in the $LJHOME directory. On Dreamhacks, this is in: ~/dw. When you make changes to the code in this directory and restart the server, your changes will show up.

You can see what changes you have made by doing git diff: the output shows all local changes to the branch you're in since the last commit.

You can revert an the original committed version of the code by [INSERT GIT INSTRUCTIONS]

Finding what you're looking for in the code

The code is a little ... sprawling. Finding what you're looking to change can be pretty hard. Fortunately, we have some instructions at Dev Finding Things.

( ...insert more about where to look for things, common use areas, how to use rgrep for fun and profit, etc )

( ...insert more about how to recover from a disaster, aka What Happens When You Break It )

Getting back to a clean slate

Sometimes when you're investigating an issue, you'll end up with a bunch of debug code hanging around in random files. To see all your changes, run:

   git status


To revert all of your changes and get back to a clean slate, run:

   git checkout .

To revert only the changes in specific files or directories, run:

   git checkout dir/subdir dir/file.txt