Difference between revisions of "Workers"

Revision as of 20:42, 15 June 2013

Workers are programs that don't run under control of Apache. They are processes which happen asynchronously, either because they use a lot of resources (e.g. search), with a contact external sites (e.g. importer).

Some workers process TheSchwartz or Gearman requests. Those normally run under control of worker-manager (see below). Some workers are manual, and depending on their purpose, can run either under control of worker-manager or be started by cron, or even just once when {re)starting the server. (Manual workers are a mixed lot, and there's no "one size fits all" option for them.)

Starting and stopping workers

Individually

When developing, you can start and debug individual workers from the shell prompt, by typing:

$LJHOME/bin/worker/(worker-name) --verbose

The --verbose or -v flag makes sure that the worker stays in the foreground and provides extra debugging text. To stop the worker when you're done, use ^C.

If you run the worker without --verbose flag, it will continue running in the background even after you're done and have logged out. You probably don't want to do this! If you want run multiple workers at the same time, or have them keep running, use the worker-manager described below.

Most of the time, if you are developing a worker, you will have a separate shell where you run the worker and can see its error messages as you work. Remember that every time you edit the code run by the worker (whether it is directly in the worker or just the libraries the worker uses) you need to kill and restart the worker process itself, or the worker will not see your changes.

Using worker-manager and workers.conf

bin/worker-manager takes care of starting, stopping, and restarting workers to maintain the numbers of workers set in etc/workers.conf for each host it runs on. The etc/workers.conf syntax is pretty straightforward and described in the file itself. To start bin/worker-manager for normal use (daemonized), use:

$LJHOME/bin/worker-manager

Or, if you want it to stay in the foreground and display progress/debug messages, use:

$LJHOME/bin/worker-manager --debug

In both cases, killing it will also kill the workers it started.

If you find that you do not want to use some workers (for testing, temporarily, or long-term), you may turn them off in the configuration. In /etc/workers.conf, setting the count to 0 will not start any of that worker. Commenting out the line on which that worker appears (with a #) will not start any of that worker. These are best for temporary changes. For a permanent change, you may also entirely remove the line on which that worker appears .

Using cron or at server boot time

Please refer to your OS documentation. The crontab(1) and init(8) manpages are probably most relevant.

List of workers

Below is a list of all the workers in the $LJHOME/bin/worker directory.

Note: I probably got some of that wrong. Any corrections welcome, especially where I say "not sure".