Dev Finding Things

Finding code corresponding to a URL

To find the code responsible for a particular page (for Template Toolkit pages), see the Routing and Template Cookbook and in particular finding the handler for a URL.

Grepping through code

Basic grep instructions that will recursively search directories for a given term:

# case insensitive
rgrep -i [search term] [path]
# case sensitive 
rgrep [search term] [path]
# example -- search for "community" in dw-nonfree
rgrep -i "community" $LJHOME/ext/dw-nonfree

To output the results of that search into a file in your home directory:

rgrep -i [search term] * > ~/filename.txt

To browse through all files containing a search term (instead of just getting a list of matching lines):

less `rgrep -il [search term] *`

An example of grep

Let's say that I want to change something in the navstrip, but I'm not sure where the navstrip lives. The easiest way to find something that you're looking at in the browser, but can't identify in the code, is to use grep, and to search for a translation string that you see on the page.

The easiest way to see what a translation string is named is to add ?uselang=debug to the end of the page URL. So, in this case, I'd go to a journal that was showing me the navstrip:

  http://zorkian.dreamwidth.org/?uselang=debug

Usually this will show you the translation strings in the wild (see http://www.dreamwidth.org/openid/?uselang=debug for instance). It doesn't seem to work on styled journal pages, though, so instead I'll pick a piece of text that looks very recognizeable, in this case the text that says "you have mutual access and mutual subscriptions". I'll do a search for that with rgrep:

rgrep -i "you have mutual access and mutual subscriptions" * | more

The output I get looks like this:

bin/upgrading/en.dat:web.controlstrip.status.mutualtrust_mutualwatch=
[[user]] and you have mutual access and mutual subscriptions

That gives me the name of the string: 'web.controlstrip.status.mutualtrust_mutualwatch'. Using that, I can then grep for it, too:

rgrep -i "web.controlstrip.status.mutualtrust_mutualwatch" * | more

which gives me:

bin/upgrading/en.dat:web.controlstrip.status.mutualtrust_mutualwatch=
[[user]] and you have mutual access and mutual subscriptions
cgi-bin/weblib.pl:                    'mutualtrust_mutualwatch' => BML::ml('web.
controlstrip.status.mutualtrust_mutualwatch', {'user' => $journal_display}),

One of them is where the string is defined; the other is where it's used. So, now I know that if I'm looking for navstrip-related code, I probably need to look in cgi-bin/weblib.pl.

Finding filenames that match a pattern

find is a powerful *nix tool for recursively searching directories. Its syntax is:

find [starting-path] [what-to-find]

So, for example:

# starting at the current directory, find files exactly named "stats.bml"
> find . -name stats.bml
./htdocs/stats.bml
 
# do a case-insensitive search for files named "user.pm"
> find . -iname user.pm
./cgi-bin/DW/External/User.pm
./cgi-bin/LJ/User.pm
 
# you can also use wildcards if you quote them --
# eg, find every .bml file that still exists
> find . -iname '*.bml'
./htdocs/tools/opml.bml
./htdocs/tools/emailmanage.bml
./htdocs/tools/userpicfactory.bml
./htdocs/tools/search.bml
./htdocs/tools/tellafriend.bml
  ...many more...
./htdocs/imgupload.bml
 
# finally, you can also use find as a recursive directory lister -- 
# entering no search terms means it shows everything
shade> find test
test
test/bin
test/bin/ipl
test/examples
test/examples/moveuclusterd_tests.pl
test/examples/example.test.pl
test/MANIFEST
  ...etc...

Directory Structure

An outline of a live site's directory structure can be found in Directory Structure.

Managing the database schema

The database schema is managed by editing update-db-general.pl. Update-db-general.pl is organized roughly like this so:

• mark which tables are clustered (versus global)
• table creation, post-create table population, table deletion
• table alteration

Functions you may need to use:

• creating a new table

register_tablecreate( "tablename", <<'EOC');
CREATE TABLE tablename (
  ....
)
EOC

• drop a table
  • remove the relevant register_tablecreate code
  • drop the table if it exists already (for older installations)

register_tabledrop( "tablename" );

• adding a column to an existing table
  • add the column to the register_tablecreate code, for new installations
  • add this to the bottom of register_alter, for existing installations

# check that the column does not yet exist
unless ( column_type( "tablename", "columname" ) ) {
   do_alter("tablename", "ALTER TABLE tablename ADD ..." );
}

• modifying a column in an existing table / modifying data in an existing table

Similar to adding a column, but instead of checking whether a table exists, you may want to check against index_name, table_relevant, column_type, e.g.,

 if ( index_name( ... ) ) {}

• initialize a table with data from another table on table creation

post_create( "tablename", /*sql statements/code. Look at other examples in the file */ );

To update your database with your changes, run

$LJHOME/bin/upgrading/update-db.pl -r --innodb

Properties

Properties are listed in bin/upgrading/proplists.dat, broken down into three sections and listed alphabetically within those sections:

1. userproplist (user properties)
2. talkproplist (comment properties)
3. logproplist (entry properties)

When removing, remove the property from the file, and also do a grep over the codebase to remove any code which uses this prop.

Translation strings

These are where all the static text gets defined. There are some oddities to working with them: see English-stripping.

Translation strings are defined in a few different files:

• bin/upgrading/en.dat - central location for strings for the backend code, e.g., things in cgi-bin, hooks, widgets
• htdocs/page.bml.text - scattered throughout htdocs; partner to htdocs/page.bml

There is a file, deadphrases.dat, which was meant for listing phrases that are no longer in use. However, everyone just forgets to use it anyway ;-) Ignore this file.

After making your changes, run

$LJHOME/bin/upgrading/texttool.pl load

to actually load the new/modified strings into your live environment.

Hooks

Hook implementations can be found in cgi-bin/LJ/Hooks.pm and cgi-bin/DW/Hooks.pm.

Some hook documentation can be found in doc/raw/hooks.txt, but it is very outdated and incomplete, and probably should be ignored right now.

Memcache

A list of memcache keys can be found in doc/raw/memcache-keys.txt. You should update this if you add new memcache keys or change existing behavior.

Styles

• styles/s2layers.dat, ext/dw-nonfree/styles/s2layers.dat

Defines system layers + layer types + layer parents.

• styles/$stylename/layout.s2, ext/dw-nonfree/styles/$stylename/layout.s2, styles/$stylename/themes.s2, ext/dw-nonfree/styles/$stylename/themes.s2

Contains actual layout and theme layers.

• cgi-bin/LJ/S2Themes.pm, ext/dw-nonfree/cgi-bin/LJ/S2Themes-local.pm

Handles some logic surrounding themes, contains list of default themes for each style, defines available subcategories for properties and sorts default ones.

• cgi-bin/LJ/S2Theme/$stylename, ext/dw-nonfree/cgi-bin/LJ/S2Theme/$stylename

Contains logic for /customize/ and /customize/options.bml pages, defines supported page set-ups and arranges custom properties into subcategories.