Difference between revisions of "Dev Getting Started"

From Dreamwidth Notes
Jump to: navigation, search
(The long version: improved references to github issues)
(gutting this and replacing it with a differently-structured version for Fu to link and flesh out)
Line 1: Line 1:
{{Cleanup|text=Adding this because I think the difference between the short section and the long section is a bit unclear (see talk page).}}
+
Welcome to Dreamwidth code development! This page is a guide to how to get started with hacking on the Dreamwidth code. It's intended to cover all steps of the process from the very beginning, but each individual step is a brief overview -- follow the links for more detail. If you're confused about anything here, you can ask us in <dwcomm>dw-dev-training</dwcomm> or come talk to us in the [irc://irc.freenode.org/dreamwidth-dev #dreamwidth-dev] [[IRC]] channel.
  
Welcome to Dreamwidth code development! This page is a guide to how to get started with hacking on the Dreamwidth code.  
+
This is geared at people who have little to no development experience. If you've got more experience and just need the details of how our project differs from other open source projects out there, check out the [[Dev Quick Start]] list.
  
== What you need ==
 
 
The languages and skills Dreamwidth development uses are:
 
The languages and skills Dreamwidth development uses are:
  
Line 17: Line 16:
  
  
To hack on the Dreamwidth code, you'll need a development environment: a place where you can install the Dreamwidth code yourself, to make changes and submit them.  We offer hosted sandbox development environments where we'll install the code for you. We call them [[Dreamhacks]], and they're available free of charge for anyone who's interested in contributing to the project. Information on how to apply can be found at the [http://hack.dreamwidth.net/ main Dreamhack page]. We also have instructions below if you want to set up your own, but be warned, it is a bit of a process.
+
== One time only: Initial setup ==
  
==The short version==
+
What you need:
  
For those who are experienced enough to not need a walkthrough:
+
=== A working Dreamwidth installation ===
  
=== Code and bug tracker ===
+
To hack on the Dreamwidth code, you'll need a development environment: a place where you can install the Dreamwidth code yourself, to make changes and submit them. We offer hosted sandbox development environments where we'll install the code for you. We call them [[Dreamhacks]], and they're available free of charge for anyone who's interested in contributing to the project. (It's possible for you to install the code yourself on your own server, but we don't recommend it for beginners. Even the most experienced Dreamwidth developers mostly work on Dreamhacks.)
* [http://github.com/dreamwidth/ Git repository on GitHub] - public repository where you can browse the Dreamwidth code. The main parts of the code you will be interested in are in <tt>dw-free</tt>
+
*  [https://github.com/dreamwidth/dw-free/issues Issue tracker (GitHub Issues)] - see also [[Github Issues]]
+
* <dwcomm>changelog</dwcomm> - changelog community on Dreamwidth which automatically posts commits
+
  
=== Development ===
+
* [http://hack.dreamwidth.net/ Apply for a Dreamhack]: this will give you a hosted development environment with everything already ready for you to get started.  
* Development environment - you have two options
+
** [http://hack.dreamwidth.net/ Apply for a Dreamhack] - get a development environment with everything set up for you (preferred)
+
** [[Dreamwidth Scratch Installation]] - set up a development environment from scratch
+
* [[Dev Finding Things]] - getting around the code base and figuring out where to start when looking for a specific feature
+
* [[Dev Programming Guidelines]] - programming style
+
* [[Design]] - things to keep in mind when designing features. Developers, please take note of this page!
+
* [[Dev Maintenance]]  - how to keep your code updated
+
* [[Version Control]] - basic commands for keeping track of your changes using version control. Also includes instructions for submitting those changes via a pull request
+
  
Don't forget your [[CLA|contributor licensing agreement]] - we'll need one of these from you before we can accept any contributions.
+
* Go through the [[Beginning dev checklist]] and follow along with the steps for getting your Dreamhack account set up. Most of these steps are tasks you'll only have to do once to make sure your development environment is set up to hack on.
  
==The long version==
+
=== A GitHub account ===
  
If you're totally new to programming and/or contributing to Open Source projects, don't worry! We'll walk you through the process.  
+
[[Git Getting Started | GitHub]] is the [[Version Control]] system we use. It keeps the "master version" of the Dreamwidth code, keeps track of the changes you make, and allows you to submit those changes to us for us to review and incorporate them.
  
Your best source of real-time information, including help for what to do when you totally break something, is the [irc://irc.freenode.org/dreamwidth-dev #dreamwidth-dev] [[IRC]] channel, where there are people willing to mentor who can figure out where your abilities are at and how to best apply them.  We also have a Dreamwidth community: <dwcomm>dw_dev_training</dwcomm>. It's a good idea to watch both that and the <dwcomm>dw_dev</dwcomm> community.
+
* Create a GitHub account with your preferred username.
  
Here's a step-by-step checklist for getting started hacking on the Dreamwidth code.
+
* Fork the dw-free and dw-nonfree repositories from the Dreamwidth repository.
  
# [http://azurelunatic.dreamwidth.org/6731755.html Bookmark the dev pep-talk that occurred in IRC in early 2012].  Also, have a look through the Epic List of [[Things Real Dreamwidth Programmers Do]].
+
===A signed CLA===
# Get a development environment. We run the [[Dreamhacks|Dreamhack]] service so you won't have to install and configure the code yourself, so all you need to do is [http://hack.dreamwidth.net/apply.shtml apply for a Dreamhack account.]
+
# Read over the [[Beginning dev checklist]] and follow along with the steps for getting your Dreamhack account set up. Most of these steps are tasks you'll only have to do once to make sure your development environment is set up to hack on.
+
# If you run into problems and have to ask for help, it would be great if you could describe the process in <dwuser>dw_dev_training</dwuser>, and ask someone with wiki privs to make a note on the [[Dev Wanted How-To]] or [[Installation Wanted How-To]] pages after you get the problem fixed, so someone can make sure to document it for someone else having the same problem. (Or ask for wiki privs yourself, either to make the note or to put in a full article!)
+
# Familiarize yourself with Perl, the language that the majority of DW development is done in. The best reference that we've found to point people at is [http://www.ebb.org/PickingUpPerl/ Picking Up Perl]. You absolutely don't need to be a Perl guru to submit a patch, but you should be familiar with the basic syntax.
+
# If you'd like to do some practice work first, check out [https://openhatch.org/missions/ OpenHatch's Training Missions] -- they'll let you practice with some of the tools you might need! (In particular, you might find their Git training mission helpful.)
+
# Browse through some of the [http://github.com/dreamwidth/dw-free Dreamwidth source code] to get a good sense of where things are and what the setup looks like. The basic directory structure, as well as some guidelines about what goes where, can be found in [[Directory Structure]].
+
# Read the [[Programming Guidelines]]. (You may not understand all of the items there, but you can familiarize yourself with them, and start noticing when existing code doesn't follow the rules as you read.)
+
# Send in a signed [[Contributor Licensing Agreement]], or CLA. We need to have that on file before we accept any of your contributions.
+
# Pick a bug that you want to work on! You'll want to look at [https://github.com/dreamwidth/dw-free/issues?q=is%3Aopen+is%3Aissue+no%3Aassignee unclaimed issues]; searches to make it easier to locate current issues are a work in progress. From time to time, people also post lists of "[http://dw-dev-training.dreamwidth.org/tag/babydev+bait babydev bait]" to <dwcomm>dw_dev_training</dwcomm>, and there is [http://dw-dev-training.dreamwidth.org/45772.html a masterlist]. These days, there's even [https://github.com/dreamwidth/dw-free/issues?q=is%3Aopen+is%3Aissue+no%3Aassignee+label%3A%22curated%3A+beginner%22 a label on GHI]!
+
# Comment on the issue you've chosen with some variation on the word "claim" ("claim", "claimed" and "claiming" all work, and can be surrounded by any other text!)
+
# Make your fixes. Be sure to test! It's a good idea to test multiple use cases, including logged in/not logged in, community vs personal journal (if applicable), personal journal with different settings, etc. For a step-by-step on this, see [[Dev_Getting_Started#Workflow|Workflow]] below.
+
# [[Version Control|Commit your changes and submit them to us]] so that they can be integrated into the official Dreamwidth code
+
# If your patch passes [[Code Review|review]], it'll be committed. If there are things that could be improved, the reviewer will comment on the commit and give you notes on what can be improved. Don't worry if your patch doesn't pass review on the first time -- it's really common for patches to go through several versions before they're committed.
+
  
=== Workflow ===
+
The [[CLA|contributor licensing agreement]] is a document that says you allow us to use your changes. You'll need to sign one before we can accept any contributions.  
Most of the links in this bulleted list go to subsections of [[Version Control]], which is a very helpful document.
+
  
# Decide what you want to work on (see [[Github Issues]]).
+
* Print a copy of the last (signature) page of the CLA and sign it
# Make sure your git repository is up to date.
+
* Scan the signed page or take a photo of it and email it to us, or mail a physical copy of it to us.
# [[Version_Control#Starting_a_new_branch|Create a new branch in git for this issue]].
+
 
# Make your changes, probably in your [[Dreamhacks|Dreamhack]] (editing text files, though the tricky part is [[Dev Finding Things|finding things]]).
+
===A basic sense of the code structure and how to find things===
# Restart Apache and test your changes on the Dreamhack.
+
 
# [[Version_Control#Committing_changes|Commit your changes to git.]]
+
You aren't going to know everything right up front, of course, but there are a few things you can look through and familiarize yourself with.
# [[Version_Control#Pushing_your_changes_to_your_repository_on_Github|Push your new git branch to your github account.]]
+
 
# [[Version_Control#Making_a_pull_request|Open a pull request.]]
+
* Familiarize yourself with Perl, the language that the majority of DW development is done in. The best reference that we've found to point people at is [http://www.ebb.org/PickingUpPerl/ Picking Up Perl]. You absolutely don't need to be a Perl guru to submit a patch, but you should be familiar with the basic syntax.
# The request will be reviewed by a senior Dreamwidth developer, and accepted or returned for further work.  
+
 
 +
* If you'd like to practice first, check out [https://openhatch.org/missions/ OpenHatch's Training Missions] -- they'll let you practice with some of the tools you might need! (In particular, you might find their Git training mission helpful.)
 +
 
 +
* Browse through some of the [http://github.com/dreamwidth/dw-free Dreamwidth source code] to get a good sense of where things are and what the setup looks like. The basic directory structure, as well as some guidelines about what goes where, can be found in [[Directory Structure]].
 +
 
 +
* Read the [[Programming Guidelines]]. (You may not understand all of the items there, but you can familiarize yourself with them, and start noticing when existing code doesn't follow the rules as you read.)
 +
 
 +
* If you start to get overwhelmed, don't worry! Go do something else for a bit and come back to this later -- this doesn't all have to be done at once. You can read an old [http://azurelunatic.dreamwidth.org/6731755.html dev pep-talk].  Also, have a look through the Epic List of [[Things Real Dreamwidth Programmers Do]] if you start feeling a bit of [https://en.wikipedia.org/wiki/Impostor_syndrome impostor syndrome]. Finally, you can always ask for help in irc or in <dwcomm>dw-dev-training</dwcomm> -- everybody who's working on the project now was in your shoes at one point, and we all remember what it's like to feel overwhelmed and over our heads. We want to help you!
 +
 
 +
* If you run into problems following any of these instructions, we also want to know about it so we can fix this documentation. You can let us know in irc or in <dwcomm>dw-dev-training</dwcomm>, or make a note on the [[Dev Wanted How-To]] or [[Installation Wanted How-To]] pages to describe what you found confusing.  
 +
 
 +
 
 +
== Finding something to work on ==
 +
 
 +
Once you've gone through the initial setup, pick an issue to work on. The issues are kept in [https://github.com/dreamwidth/dw-free/issues?q=is%3Aopen GitHub Issues].
 +
 
 +
* Look through the [https://github.com/dreamwidth/dw-free/labels/curated:%20beginner curated:beginner] label for issues that someone thinks would make a good first issue to work on. From time to time, people also post lists of "[http://dw-dev-training.dreamwidth.org/tag/babydev+bait babydev bait]" to <dwcomm>dw_dev_training</dwcomm>, and there is [http://dw-dev-training.dreamwidth.org/45772.html a masterlist].
 +
 
 +
* Let people know that you're working on the bug by assigning it to yourself. Comment on the issue you've chosen with some variation on the word "claim" ("claim", "claimed" and "claiming" all work, and can be surrounded by any other text). This will trigger our bot to assign the issue to you.
 +
 
 +
* [[Version_Control#Starting_a_new_branch|Create a new branch in git for this issue]] so you have a place to make your changes.
 +
 
 +
== Making your changes ==
 +
 
 +
* Make sure your git repository is up to date.
 +
 
 +
* Make your changes, probably in your [[Dreamhacks|Dreamhack]] (editing text files, though the tricky part is [[Dev Finding Things|finding things]]).
 +
 
 +
* Restart Apache and test your changes on the Dreamhack. It's a good idea to test multiple use cases, including logged in/not logged in, community vs personal journal (if applicable), personal journal with different settings, etc.  
 +
 
 +
* [[Version_Control#Committing_changes|Commit your changes to git.]]
 +
 
 +
* [[Version_Control#Pushing_your_changes_to_your_repository_on_Github|Push your new git branch to your github account.]]
 +
 
 +
 
 +
== Submitting your changes ==
 +
 
 +
* Once you're satisfied that your changes fix the issue and don't introduce any new bugs, [[Version_Control#Making_a_pull_request|open a pull request.]] This sends your changes to us for us to look over in [[Code Review|review]].
 +
 
 +
* A senior Dreamwidth developer will look over your changes and see if they can be accepted or if there are things you need to change. If there are things that could be improved, the reviewer will comment on the commit and give you notes on what can be improved. Don't worry if your changes don't pass review on the first time -- it's really common for pull requests to go through several versions before they're committed.
 +
 
 +
* If your pull request needs more changes, you can go back to the branch you made for the change and make those changes. Once you commit them to your branch, the pull request will update automatically.
 +
 
 +
* When your pull request is accepted, the person who accepts it will close the pull request and the issue that the pull request fixed.
 +
 
 +
 
 +
== The next fix! ==
 +
 
 +
* Once your pull request is accepted and merged into the main branch, you can delete the branch for that issue on your GitHub account.
 +
 
 +
* Switch back to the 'develop' branch on your Dreamhack and update to the most current code.
 +
 
 +
* PIck the next issue to work on!
 +
 
 +
== Further reading ==
 +
 
 +
This is a brief overview of the process. There are plenty of places to go for more information. Useful wiki pages include:
 +
 
 +
* [[Dev Finding Things]] - getting around the code base and figuring out where to start when looking for a specific feature
 +
* [[Dev Programming Guidelines]] - programming style
 +
* [[Design]] - things to keep in mind when designing features. Developers, please take note of this page!
 +
* [[Dev Maintenance]]  - how to keep your code updated
 +
* [[Version Control]] - basic commands for keeping track of your changes using version control. Also includes instructions for submitting those changes via a pull request
 +
* [[Development]] - another list of stuff that might be useful for you at some point
 +
* [[Git Getting Started]] - a collection of resources for using Git
 +
* [[:Category:Development]] - every page tagged with the Development category on the wiki.
  
See [[Development]] and [[Git Getting Started]] for a more complete reading list, or look through the pages in [[:Category:Development]].
 
  
 
==What'd we miss?==
 
==What'd we miss?==

Revision as of 06:58, 14 October 2014

Welcome to Dreamwidth code development! This page is a guide to how to get started with hacking on the Dreamwidth code. It's intended to cover all steps of the process from the very beginning, but each individual step is a brief overview -- follow the links for more detail. If you're confused about anything here, you can ask us in [info]dw-dev-training or come talk to us in the #dreamwidth-dev IRC channel.

This is geared at people who have little to no development experience. If you've got more experience and just need the details of how our project differs from other open source projects out there, check out the Dev Quick Start list.

The languages and skills Dreamwidth development uses are:

  • Perl (the majority of the project)
  • BML (custom templating language inherited from LiveJournal which we are currently phasing out)
  • TT (Template Toolkit - better templating language, used for new pages)
  • S2 (custom templating language used in journal styles)
  • HTML / CSS
  • Javascript / JQuery
  • MySQL and DBI (the Perl interface to MySQL and other SQL databases)

You don't need to know them all! Some familiarity with at least one will be helpful, but we welcome people at all levels, and are happy to answer questions and give advice as long as you're willing to learn.


One time only: Initial setup

What you need:

A working Dreamwidth installation

To hack on the Dreamwidth code, you'll need a development environment: a place where you can install the Dreamwidth code yourself, to make changes and submit them. We offer hosted sandbox development environments where we'll install the code for you. We call them Dreamhacks, and they're available free of charge for anyone who's interested in contributing to the project. (It's possible for you to install the code yourself on your own server, but we don't recommend it for beginners. Even the most experienced Dreamwidth developers mostly work on Dreamhacks.)

  • Apply for a Dreamhack: this will give you a hosted development environment with everything already ready for you to get started.
  • Go through the Beginning dev checklist and follow along with the steps for getting your Dreamhack account set up. Most of these steps are tasks you'll only have to do once to make sure your development environment is set up to hack on.

A GitHub account

GitHub is the Version Control system we use. It keeps the "master version" of the Dreamwidth code, keeps track of the changes you make, and allows you to submit those changes to us for us to review and incorporate them.

  • Create a GitHub account with your preferred username.
  • Fork the dw-free and dw-nonfree repositories from the Dreamwidth repository.

A signed CLA

The contributor licensing agreement is a document that says you allow us to use your changes. You'll need to sign one before we can accept any contributions.

  • Print a copy of the last (signature) page of the CLA and sign it
  • Scan the signed page or take a photo of it and email it to us, or mail a physical copy of it to us.

A basic sense of the code structure and how to find things

You aren't going to know everything right up front, of course, but there are a few things you can look through and familiarize yourself with.

  • Familiarize yourself with Perl, the language that the majority of DW development is done in. The best reference that we've found to point people at is Picking Up Perl. You absolutely don't need to be a Perl guru to submit a patch, but you should be familiar with the basic syntax.
  • If you'd like to practice first, check out OpenHatch's Training Missions -- they'll let you practice with some of the tools you might need! (In particular, you might find their Git training mission helpful.)
  • Browse through some of the Dreamwidth source code to get a good sense of where things are and what the setup looks like. The basic directory structure, as well as some guidelines about what goes where, can be found in Directory Structure.
  • Read the Programming Guidelines. (You may not understand all of the items there, but you can familiarize yourself with them, and start noticing when existing code doesn't follow the rules as you read.)
  • If you start to get overwhelmed, don't worry! Go do something else for a bit and come back to this later -- this doesn't all have to be done at once. You can read an old dev pep-talk. Also, have a look through the Epic List of Things Real Dreamwidth Programmers Do if you start feeling a bit of impostor syndrome. Finally, you can always ask for help in irc or in [info]dw-dev-training -- everybody who's working on the project now was in your shoes at one point, and we all remember what it's like to feel overwhelmed and over our heads. We want to help you!
  • If you run into problems following any of these instructions, we also want to know about it so we can fix this documentation. You can let us know in irc or in [info]dw-dev-training, or make a note on the Dev Wanted How-To or Installation Wanted How-To pages to describe what you found confusing.


Finding something to work on

Once you've gone through the initial setup, pick an issue to work on. The issues are kept in GitHub Issues.

  • Let people know that you're working on the bug by assigning it to yourself. Comment on the issue you've chosen with some variation on the word "claim" ("claim", "claimed" and "claiming" all work, and can be surrounded by any other text). This will trigger our bot to assign the issue to you.

Making your changes

  • Make sure your git repository is up to date.
  • Restart Apache and test your changes on the Dreamhack. It's a good idea to test multiple use cases, including logged in/not logged in, community vs personal journal (if applicable), personal journal with different settings, etc.


Submitting your changes

  • Once you're satisfied that your changes fix the issue and don't introduce any new bugs, open a pull request. This sends your changes to us for us to look over in review.
  • A senior Dreamwidth developer will look over your changes and see if they can be accepted or if there are things you need to change. If there are things that could be improved, the reviewer will comment on the commit and give you notes on what can be improved. Don't worry if your changes don't pass review on the first time -- it's really common for pull requests to go through several versions before they're committed.
  • If your pull request needs more changes, you can go back to the branch you made for the change and make those changes. Once you commit them to your branch, the pull request will update automatically.
  • When your pull request is accepted, the person who accepts it will close the pull request and the issue that the pull request fixed.


The next fix!

  • Once your pull request is accepted and merged into the main branch, you can delete the branch for that issue on your GitHub account.
  • Switch back to the 'develop' branch on your Dreamhack and update to the most current code.
  • PIck the next issue to work on!

Further reading

This is a brief overview of the process. There are plenty of places to go for more information. Useful wiki pages include:

  • Dev Finding Things - getting around the code base and figuring out where to start when looking for a specific feature
  • Dev Programming Guidelines - programming style
  • Design - things to keep in mind when designing features. Developers, please take note of this page!
  • Dev Maintenance - how to keep your code updated
  • Version Control - basic commands for keeping track of your changes using version control. Also includes instructions for submitting those changes via a pull request
  • Development - another list of stuff that might be useful for you at some point
  • Git Getting Started - a collection of resources for using Git
  • Category:Development - every page tagged with the Development category on the wiki.


What'd we miss?

If you run into anything in the process that isn't well-documented, or you have questions, you can ask in the [info]dw_dev_training community or in the #dreamwidth-dev irc channel. You should also make a note on the Dev Wanted How-To or Installation Wanted How-To wiki pages so we can document it better for the next person to come along!