Difference between revisions of "ESN"

From Dreamwidth Notes
Jump to: navigation, search
(Firing events and delivering notifications: Flesh out section)
m (Events: Proper punctuation order)
 
(12 intermediate revisions by 3 users not shown)
Line 5: Line 5:
 
An event is anything happening on Dreamwidth that a user may want to be notified of when it happens, like "someone replied to a comment I wrote" or "there's a new entry posted in (journal)".
 
An event is anything happening on Dreamwidth that a user may want to be notified of when it happens, like "someone replied to a comment I wrote" or "there's a new entry posted in (journal)".
  
Events are defined in [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/Event.pm LJ::Event] (base class) and [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/Event LJ::Event::*] (subclasses for specific event types). They have some data attached, including the journal the event is about and up to 2 additional numeric arguments. See the comments at the beginning of [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/Event.pm LJ::Event] for more on event types, subclasses, and what the journal and additional arguments are used for.
+
Events are defined in [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/Event.pm LJ::Event] (base class) and [https://github.com/dreamwidth/dw-free/tree/develop/cgi-bin/LJ/Event LJ::Event::*] (subclasses for specific event types). They have some data attached, including the journal the event is about and up to 2 additional numeric arguments. See [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/Event.pm#L30 the comments at the beginning of LJ::Event] for more on event types, subclasses, and what the journal and additional arguments are used for.
 +
 
 +
Each event class is responsible for providing the notification email subject, plain text, and HTML text, by implementing the <code>as_email_subject</code>, <code>as_email_string</code>, and <code>as_email_html</code> methods. These methods are called by <code>LJ::NotificationMethod::Email</code> (see below) as needed. (as_email_html won't get called if the user doesn't want HTML email.)
  
 
== Subscriptions ==
 
== Subscriptions ==
Line 11: Line 13:
 
Subscriptions are how someone tells Dreamwidth which events they're interested in. Which events or event types a user may subscribe to may be restricted by the user's account level, but beyond that, any user can subscribe to any available event if they have access to the relevant data (eg, can view the journal/entry/birthdate/etc...).
 
Subscriptions are how someone tells Dreamwidth which events they're interested in. Which events or event types a user may subscribe to may be restricted by the user's account level, but beyond that, any user can subscribe to any available event if they have access to the relevant data (eg, can view the journal/entry/birthdate/etc...).
  
Subscriptions are defined in [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/Subscription.pm LJ::Subscription] (base class) and [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/Subscription/Pending.pm LJ::Subscription::Pending] (used by the user interface for things the user hasn't (yet) subscribed to). Unlike for events, there are no subclasses for different things subscribed to. Data for each subscription includes the journal of interest, the type of event, the notification method (see below), and 2 additional (numerical) arguments, which aren't necessarily equal or even related to the ones for events that may match the suggestion.
+
Subscriptions are defined in [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/Subscription.pm LJ::Subscription] (base class) and [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/Subscription/Pending.pm LJ::Subscription::Pending] (used by the user interface for things the user hasn't (yet) subscribed to). Unlike for events, there are no subclasses for different things subscribed to. Data for each subscription includes the journal of interest, the type of event, the notification method (see below), and 2 additional (numerical) arguments, which aren't necessarily equal or even related to the ones for events that may match the subscription.
  
 
== Notification methods ==
 
== Notification methods ==
  
Notification methods are the possible ways you can get notified when an event that matches one of your subscriptions occurs. They're implemented in [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/NotificationMethod.pm LJ::NotificationMethod] (base class) and classes for specific notification methods:
+
Notification methods are the possible ways you can get notified when an event that matches one of your subscriptions occurs. They're implemented in [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/NotificationMethod.pm LJ::NotificationMethod] (base class) and classes for specific notification methods:
  
* [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/NotificationMethod/Email.pm LJ::NotificationMethod::Email] notifies users by sending them email.
+
* [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/NotificationMethod/Email.pm LJ::NotificationMethod::Email] notifies users by sending them email. It handles both plain-text and HTML email as appropriate, by calling the event's <code>as_email_subject</code>, <code>as_email_string</code>, and <code>as_email_html</code> methods. (There are hooks available, but Dreamwidth doesn't use them. See [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/NotificationMethod/Email.pm#L66 LJ::NotificationMethod::Email->notify] for more details.)
  
* [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/NotificationMethod/Inbox.pm LJ::NotificationMethod::Inbox] notifies users by placing a message in their site inbox.
+
* [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/NotificationMethod/Inbox.pm LJ::NotificationMethod::Inbox] notifies users by placing a message in their site inbox.
  
* [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/NotificationMethod/IM.pm LJ::NotificationMethod::IM] isn't offered to users (yet) because Dreamwidth doesn't have the infrastructure to send Jabber messages. (Which is all it supports. No AIM or anything of the kind.)
+
* [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/NotificationMethod/DebugLog.pm LJ::NotificationMethod::DebugLog] isn't offered to users, because it doesn't actually send them anything. Instead, it saves all notification data to a database table, which is useful for developers. See <code>LJ::NotificationMethod::DebugLog->notify</code> for more.
 
+
* [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/NotificationMethod/DebugLog.pm LJ::NotificationMethod::DebugLog] isn't offered to users either, because it doesn't actually send them anything. Instead, it saves all notification data to a database table, which is useful for developers. See LJ::NotificationMethod::DebugLog->notify for more.
+
  
 
== Firing events and delivering notifications ==
 
== Firing events and delivering notifications ==
  
To fire an event, use LJ::Event->fire_job. This queues a LJ::Worker::FiredEvent [[TheSchwartz]] job. The bulk of processing fired events takes place in [http://hg.dwscoalition.org/dw-free/file/tip/cgi-bin/LJ/ESN.pm LJ::ESN], in 4 stages:
+
To fire an event, use <code>LJ::Event->fire_job</code>. This queues a <code>LJ::Worker::FiredEvent</code> [[TheSchwartz]] job. The bulk of processing fired events takes place in [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/LJ/ESN.pm LJ::ESN], in 4 stages:
  
# LJ::Worker::FiredEvent finds relevant subscriptions, either by queuing a LJ::Worker::FindSubsByCluster subjob for each cluster (see next stage), or (if there aren't too many subscriptions and no cluster is down) by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
+
# <code>LJ::Worker::FiredEvent</code> finds relevant subscriptions, either by queuing a <code>LJ::Worker::FindSubsByCluster</code> subjob for each cluster (see next stage), or (if there aren't too many subscriptions and no cluster is down) by directly removing duplicate or redundant subscriptions and skipping to the last stage, <code>LJ::Worker::ProcessSub</code>.
 +
# <code>LJ::Worker::FindSubsByCluster</code> finds relevant subscriptions for a specified cluster, either by queuing a <code>LJ::Worker::FilterSubs</code> job for each group of up to <code>$LJ::ESN::MAX_FILTER_SET</code> (currently 5000) that doesn't split subscriptions by the same user, or by directly removing duplicate or redundant subscriptions and skipping to the last stage, <code>LJ::Worker::ProcessSub</code>.
 +
# <code>LJ::Worker::FilterSubs</code> checks for race conditions (user halfway through cluster move, with some subscriptions left on old cluster), then removes duplicate or redundant subscriptions and passes the remaining ones to the last stage, by queuing one job for each.
 +
# <code>LJ::Worker::ProcessSub</code> processes a subscription by creating a notification of the appropriate subclass and having it talk to the user.
  
# LJ::Worker::FindSubsByCluster finds relevant subscriptions for a specified cluster, either by queuing a LJ::Worker::FilterSubs job for each group of up to $LJ::ESN::MAX_FILTER_SET (currently 5000) that doesn't split subscriptions by the same user, or by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
+
Some environment variables and testing flags are available to prevent skipping stages 2 or 3: <code>$ENV{FORCE_P1_P2} || $LJ::_T_ESN_FORCE_P1_P2</code> (stage 2), <code>$ENV{FORCE_P2_P3} || $LJ::_T_ESN_FORCE_P2_P3</code> (stage 3). Note that setting the later and not the former may result in stage 3 being skipped, if stage 1 skips stage 2. Also, <code>$ENV{DEBUG}</code> forces output of trace messages. (All of those have to be set to a true value to work - it's not enough for the environment variables to exist or the testing flags to have a defined value.)
  
# LJ::Worker::FilterSubs checks for race conditions (user halfway through cluster move, with some subscriptions left on old cluster), then removes duplicate or redundant subscriptions and passes the remaining ones to the last stage, by queuing one job for each.
+
There are 2 sets of workers for processing events. The first set includes:
  
# LJ::Worker::ProcessSub processes a subscription by creating a notification of the appropriate subclass and having it talk to the user.
+
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-fired-event bin/worker/esn-fired-event],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-cluster-subs bin/worker/esn-cluster-subs],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-filter-subs bin/worker/esn-filter-subs],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-process-sub bin/worker/esn-process-sub].
  
Some environment variables and testing flags are available to prevent skipping stages 2 or 3: $ENV{FORCE_P1_P2} || $LJ::_T_ESN_FORCE_P1_P2 (stage 2), $ENV{FORCE_P2_P3} || $LJ::_T_ESN_FORCE_P2_P3 (stage 3). Note that setting the later and not the former may result in stage 3 being skipped, if stage 1 skips stage 2. Also, $ENV{DEBUG} forces output of trace messages. (All of those have to be set to a true value to work - it's not enough for the environment variables to exist or the testing flags to have a defined value.)
+
The second set includes:
  
There are 2 sets of workers for processing events:
+
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-fired-event-mass bin/worker/esn-fired-event-mass],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-cluster-subs-mass bin/worker/esn-cluster-subs-mass],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-filter-subs-mass bin/worker/esn-filter-subs-mass],
 +
* [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/esn-process-sub-mass bin/worker/esn-process-sub-mass].
  
* [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-fired-event bin/worker/esn-fired-event], [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-cluster-subs bin/worker/esn-cluster-subs], [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-filter-subs bin/worker/esn-filter-subs], and [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-process-sub bin/worker/esn-process-sub]
+
Both set of workers handle the same events. The only difference is that the -mass workers use the "mass" role for accessing the queue tables, thus allowing tuning specific to huge, infrequent events like official newsposts.
  
* [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-fired-event-mass bin/worker/esn-fired-event-mass], [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-cluster-subs-mass bin/worker/esn-cluster-subs-mass], [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-filter-subs-mass bin/worker/esn-filter-subs-mass], and [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/esn-process-sub-mass bin/worker/esn-process-sub-mass]
+
If your site doesn't generate a lot of events (which would most likely be the case for a developer setup), you may want to use [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/process-esn bin/worker/process-esn] instead of the first set and [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/process-esn-mass bin/worker/process-esn-mass] instead of the second set. Each of those replaces all 4 in the corresponding set.
  
Both set of workers handle the same events. The only difference is that the -mass workers use the "mass" role for accessing the queue tables, thus allowing tuning specific to huge, infrequent events like official newsposts.
+
== Previewing notifications ==
 +
 
 +
On a development server or Dreamhack (one that has <code>$LJ::IS_DEV_SERVER</code> set to a true value, usually 1), it's possible to see what a notification email or inbox message would look like by using <nowiki>http://www.testuser.hack.dreamwidth.net/admin/eventoutput</nowiki> (replacing www.testuser.hack.dreamwidth.net with your server's domain name).
  
 
[[Category: Development]]
 
[[Category: Development]]

Latest revision as of 19:05, 23 April 2016

ESN stands for Event-Subscription-Notification. This is a complex system, and this page only scratches the surface. You should be prepared to do some digging on your own or ask questions. If you do, please think of the next person having the same question and revise this to include the information you were looking for.

Events

An event is anything happening on Dreamwidth that a user may want to be notified of when it happens, like "someone replied to a comment I wrote" or "there's a new entry posted in (journal)".

Events are defined in LJ::Event (base class) and LJ::Event::* (subclasses for specific event types). They have some data attached, including the journal the event is about and up to 2 additional numeric arguments. See the comments at the beginning of LJ::Event for more on event types, subclasses, and what the journal and additional arguments are used for.

Each event class is responsible for providing the notification email subject, plain text, and HTML text, by implementing the as_email_subject, as_email_string, and as_email_html methods. These methods are called by LJ::NotificationMethod::Email (see below) as needed. (as_email_html won't get called if the user doesn't want HTML email.)

Subscriptions

Subscriptions are how someone tells Dreamwidth which events they're interested in. Which events or event types a user may subscribe to may be restricted by the user's account level, but beyond that, any user can subscribe to any available event if they have access to the relevant data (eg, can view the journal/entry/birthdate/etc...).

Subscriptions are defined in LJ::Subscription (base class) and LJ::Subscription::Pending (used by the user interface for things the user hasn't (yet) subscribed to). Unlike for events, there are no subclasses for different things subscribed to. Data for each subscription includes the journal of interest, the type of event, the notification method (see below), and 2 additional (numerical) arguments, which aren't necessarily equal or even related to the ones for events that may match the subscription.

Notification methods

Notification methods are the possible ways you can get notified when an event that matches one of your subscriptions occurs. They're implemented in LJ::NotificationMethod (base class) and classes for specific notification methods:

  • LJ::NotificationMethod::Email notifies users by sending them email. It handles both plain-text and HTML email as appropriate, by calling the event's as_email_subject, as_email_string, and as_email_html methods. (There are hooks available, but Dreamwidth doesn't use them. See LJ::NotificationMethod::Email->notify for more details.)
  • LJ::NotificationMethod::DebugLog isn't offered to users, because it doesn't actually send them anything. Instead, it saves all notification data to a database table, which is useful for developers. See LJ::NotificationMethod::DebugLog->notify for more.

Firing events and delivering notifications

To fire an event, use LJ::Event->fire_job. This queues a LJ::Worker::FiredEvent TheSchwartz job. The bulk of processing fired events takes place in LJ::ESN, in 4 stages:

  1. LJ::Worker::FiredEvent finds relevant subscriptions, either by queuing a LJ::Worker::FindSubsByCluster subjob for each cluster (see next stage), or (if there aren't too many subscriptions and no cluster is down) by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
  2. LJ::Worker::FindSubsByCluster finds relevant subscriptions for a specified cluster, either by queuing a LJ::Worker::FilterSubs job for each group of up to $LJ::ESN::MAX_FILTER_SET (currently 5000) that doesn't split subscriptions by the same user, or by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
  3. LJ::Worker::FilterSubs checks for race conditions (user halfway through cluster move, with some subscriptions left on old cluster), then removes duplicate or redundant subscriptions and passes the remaining ones to the last stage, by queuing one job for each.
  4. LJ::Worker::ProcessSub processes a subscription by creating a notification of the appropriate subclass and having it talk to the user.

Some environment variables and testing flags are available to prevent skipping stages 2 or 3: $ENV{FORCE_P1_P2} || $LJ::_T_ESN_FORCE_P1_P2 (stage 2), $ENV{FORCE_P2_P3} || $LJ::_T_ESN_FORCE_P2_P3 (stage 3). Note that setting the later and not the former may result in stage 3 being skipped, if stage 1 skips stage 2. Also, $ENV{DEBUG} forces output of trace messages. (All of those have to be set to a true value to work - it's not enough for the environment variables to exist or the testing flags to have a defined value.)

There are 2 sets of workers for processing events. The first set includes:

The second set includes:

Both set of workers handle the same events. The only difference is that the -mass workers use the "mass" role for accessing the queue tables, thus allowing tuning specific to huge, infrequent events like official newsposts.

If your site doesn't generate a lot of events (which would most likely be the case for a developer setup), you may want to use bin/worker/process-esn instead of the first set and bin/worker/process-esn-mass instead of the second set. Each of those replaces all 4 in the corresponding set.

Previewing notifications

On a development server or Dreamhack (one that has $LJ::IS_DEV_SERVER set to a true value, usually 1), it's possible to see what a notification email or inbox message would look like by using http://www.testuser.hack.dreamwidth.net/admin/eventoutput (replacing www.testuser.hack.dreamwidth.net with your server's domain name).