Difference between revisions of "TheSchwartz"
Foxfirefey (Talk | contribs) (Created page with 'TheSchwartz is a worker manager system that runs things like the importer, the private messaging function, the crossposter, and all of the event notifications. It makes it so th…') |
(Add info about using it from Dreamwidth code) |
||
| Line 1: | Line 1: | ||
TheSchwartz is a worker manager system that runs things like the importer, the private messaging function, the crossposter, and all of the event notifications. It makes it so that the web servers don't have to spend CPU timing doing work that doesn't have to be done instantly. | TheSchwartz is a worker manager system that runs things like the importer, the private messaging function, the crossposter, and all of the event notifications. It makes it so that the web servers don't have to spend CPU timing doing work that doesn't have to be done instantly. | ||
| − | + | == Setting up TheSchwartz == | |
| − | [[ | + | See [[TheSchwartz Setup]]. |
| + | |||
| + | == Queueing a job for TheSchwartz from Dreamwidth == | ||
| + | |||
| + | === Generic Job === | ||
| + | |||
| + | The following example queues a pony-feeding job, passing it amounts of hay and oats to use. | ||
| + | |||
| + | <syntaxhighlighting lang="perl"> | ||
| + | use TheSchwartz; | ||
| + | my $sclient = LJ::theschwartz(); # LJ::theschwartz( { role => "mass" } ) for very large stables | ||
| + | die "Can't get TheSchwartz client" unless $sclient; | ||
| + | my $job = TheSchwartz::Job->new_from_array("DW::Worker::FeedPonies", | ||
| + | [ hay => $hay, oats => $oats ]); | ||
| + | die "Can't create job" unless $job; | ||
| + | $sclient->insert($job) or die "Can't queue feeding job"; | ||
| + | </syntaxhighlighting> | ||
| + | |||
| + | <code>LJ::theschwartz</code> is defined in [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/ljlib.pl cgi-bin/ljlib.pl], so if you're not in web context (command-line program, Gearman worker, TheSchwartz worker), you'll need to <code>require "ljlib.pl";</code> as well. | ||
| + | |||
| + | You should be able to adapt this to your specific job and data. | ||
| + | |||
| + | === Notifying users of ESN events they subscribed to === | ||
| + | |||
| + | Don't use the above. See [[ESN#Firing_events_and_delivering_notifications|Firing ESN events and delivering notifications]] instead. | ||
| + | |||
| + | === Sending non-ESN email to a user === | ||
| + | |||
| + | Use LJ::send_mail from [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/ljmail.pl cgi-bin/ljmail.pl]. Note that doing so correctly is fiddly, if you want to respect all of the user's email-related preferences. See [http://bugs.dwscoalition.org/show_bug.cgi?id=3946 bug 3946] for more. | ||
| + | |||
| + | == Writing a worker for Dreamwidth TheSchwartz jobs == | ||
| + | |||
| + | The following implements the <code>DW::Worker::FeedPonies</code> worker used in the example above: | ||
| + | |||
| + | <syntaxhighlighting lang="perl"> | ||
| + | use LJ::Worker::TheSchwartz; | ||
| + | schwartz_decl( "DW::Worker::FeedPonies" ); | ||
| + | schwartz_work(); # Never returns. | ||
| + | |||
| + | package DW::Worker::FeedPonies; | ||
| + | use base 'TheSchwartz::Worker'; | ||
| + | sub schwartz_capabilities { return ( __PACKAGE__ ); } | ||
| + | sub max_retries { 5 } # Try feeding ponies 5 times before giving up | ||
| + | sub grab_for { 600 } # Give the stable hand 600 seconds (10 minutes) to finish | ||
| + | sub keep_exit_status_for { 86400 } # Keep the result of the feeding attempt for 24 hours | ||
| + | |||
| + | sub retry_delay { # How long to wait before retrying on after an attempt fails | ||
| + | my ($class, $fails) = @_; | ||
| + | |||
| + | return (10, 30, 60, 300, 600)[$fails]; | ||
| + | } | ||
| + | |||
| + | sub work { # Do the actual work | ||
| + | my ($class, $job) = @_; | ||
| + | my %arg = %{$job->arg}; | ||
| + | |||
| + | $job->permanent_failure( "Amount of hay and or oats negative" ) | ||
| + | if $arg{hay} < 0 || $arg{oats} < 0; | ||
| + | $job->debug( "Feeding the ponies $arg{hay} bales of hay and $arg{oats} bushels of oats" ); | ||
| + | Ponies->feed( %arg ) or return $job->failed( "Unable to feed ponies, retrying" ); | ||
| + | |||
| + | $job->completed; | ||
| + | } | ||
| + | </syntaxhighlighting> | ||
| + | |||
| + | You should be able to adapt this to your specific job and data. | ||
| + | |||
| + | [[Category: Development]] | ||
Revision as of 18:18, 19 September 2011
TheSchwartz is a worker manager system that runs things like the importer, the private messaging function, the crossposter, and all of the event notifications. It makes it so that the web servers don't have to spend CPU timing doing work that doesn't have to be done instantly.
Contents
Setting up TheSchwartz
See TheSchwartz Setup.
Queueing a job for TheSchwartz from Dreamwidth
Generic Job
The following example queues a pony-feeding job, passing it amounts of hay and oats to use.
<syntaxhighlighting lang="perl"> use TheSchwartz; my $sclient = LJ::theschwartz(); # LJ::theschwartz( { role => "mass" } ) for very large stables die "Can't get TheSchwartz client" unless $sclient; my $job = TheSchwartz::Job->new_from_array("DW::Worker::FeedPonies",
[ hay => $hay, oats => $oats ]);
die "Can't create job" unless $job; $sclient->insert($job) or die "Can't queue feeding job"; </syntaxhighlighting>
LJ::theschwartz is defined in cgi-bin/ljlib.pl, so if you're not in web context (command-line program, Gearman worker, TheSchwartz worker), you'll need to require "ljlib.pl"; as well.
You should be able to adapt this to your specific job and data.
Notifying users of ESN events they subscribed to
Don't use the above. See Firing ESN events and delivering notifications instead.
Sending non-ESN email to a user
Use LJ::send_mail from cgi-bin/ljmail.pl. Note that doing so correctly is fiddly, if you want to respect all of the user's email-related preferences. See bug 3946 for more.
Writing a worker for Dreamwidth TheSchwartz jobs
The following implements the DW::Worker::FeedPonies worker used in the example above:
<syntaxhighlighting lang="perl"> use LJ::Worker::TheSchwartz; schwartz_decl( "DW::Worker::FeedPonies" ); schwartz_work(); # Never returns.
package DW::Worker::FeedPonies; use base 'TheSchwartz::Worker'; sub schwartz_capabilities { return ( __PACKAGE__ ); } sub max_retries { 5 } # Try feeding ponies 5 times before giving up sub grab_for { 600 } # Give the stable hand 600 seconds (10 minutes) to finish sub keep_exit_status_for { 86400 } # Keep the result of the feeding attempt for 24 hours
sub retry_delay { # How long to wait before retrying on after an attempt fails
my ($class, $fails) = @_;
return (10, 30, 60, 300, 600)[$fails];
}
sub work { # Do the actual work
my ($class, $job) = @_;
my %arg = %{$job->arg};
$job->permanent_failure( "Amount of hay and or oats negative" )
if $arg{hay} < 0 || $arg{oats} < 0;
$job->debug( "Feeding the ponies $arg{hay} bales of hay and $arg{oats} bushels of oats" );
Ponies->feed( %arg ) or return $job->failed( "Unable to feed ponies, retrying" );
$job->completed;
} </syntaxhighlighting>
You should be able to adapt this to your specific job and data.
