Difference between revisions of "Stats Design"

From Dreamwidth Notes
Jump to: navigation, search
(Wikifi format, expand statistics collection section)
m (Fix typos)
Line 5: Line 5:
 
[[Project_Teams#Site_statistics|Stats project team]] (we're recruiting)
 
[[Project_Teams#Site_statistics|Stats project team]] (we're recruiting)
  
−
== =Table definitions ==
+
== Table definitions ==
  
 
<source lang="sql">CREATE TABLE statkeylist (
 
<source lang="sql">CREATE TABLE statkeylist (
Line 38: Line 38:
 
== Collecting statistics ==
 
== Collecting statistics ==
  
−
# [http://hg.dwscoalition.org/dw-free/file/tip/etc/stats-collection.conf etc/stats-collection.conf] should contain something like:<source>## collect and store all stats under the accounts category
+
# [http://hg.dwscoalition.org/dw-free/file/tip/etc/stats-collection.conf etc/stats-collection.conf] should contain something like:<source lang="text">## collect and store all stats under the accounts category
 
## hangs if you don't enclose the asterisk in quotes
 
## hangs if you don't enclose the asterisk in quotes
 
# accounts: "*"
 
# accounts: "*"
Line 47: Line 47:
 
paid: "*"</source>
 
paid: "*"</source>
 
# [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/stats-collection bin/worker/stats-collection] is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in [http://hg.dwscoalition.org/dw-free/file/tip/etc/workers.conf etc/workers.conf].
 
# [http://hg.dwscoalition.org/dw-free/file/tip/bin/worker/stats-collection bin/worker/stats-collection] is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in [http://hg.dwscoalition.org/dw-free/file/tip/etc/workers.conf etc/workers.conf].
−
# (only needed if you chose to use [http://hg.dwscoalition.org/dw-free/file/tip/etc/workers.conf etc/workers.conf]): add a line to the "all" section reading:<source>  stats-collection: 1</source>(note: this is for development only, as it assumes you only have one server - in production environments, you will usually list it in the section for a specific server)
+
# (only needed if you chose to use [http://hg.dwscoalition.org/dw-free/file/tip/etc/workers.conf etc/workers.conf]): add a line to the "all" section reading:<source lang="text">  stats-collection: 1</source>(note: this is for development only, as it assumes you only have one server - in production environments, you will usually list it in the section for a specific server)
  
 
== Displaying statistics ==
 
== Displaying statistics ==

Revision as of 17:50, 30 March 2010

This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.)

Bug 124

Stats project team (we're recruiting)

Table definitions

CREATE TABLE statkeylist (
    statkeyid INT UNSIGNED NOT NULL AUTO_INCREMENT, 
    name VARCHAR(255) DEFAULT NULL, 
 
    PRIMARY KEY (statkeyid), 
    UNIQUE KEY (name) 
)
 
CREATE TABLE site_stats ( 
    category_id INT UNSIGNED NOT NULL, 
    key_id INT UNSIGNED NOT NULL, 
    insert_time INT UNSIGNED NOT NULL, 
    VALUE INT UNSIGNED NOT NULL, 
 
    -- FIXME: This is good for retrieving data for a single category+key, but 
    -- maybe not as good if we want all keys for the category, with a limit on 
    -- time (ie, last 5 entries, or last 2 weeks). Do we need an extra index? 
    INDEX (category_id, key_id, insert_time) 
)

Storing statistics

DW::StatStore

Statistics items

Collecting statistics

  1. etc/stats-collection.conf should contain something like:
    ## collect and store all stats under the accounts category

      

    1.  hangs if you don't enclose the asterisk in quotes

      2.
  2. accounts: "*"
    1. collect and store only these stats under the accounts category
  3. accounts: [ identity, personal, redirect ]

accounts: "*" active: "*"

paid: "*"
  1. bin/worker/stats-collection is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in etc/workers.conf.
  2. (only needed if you chose to use etc/workers.conf): add a line to the "all" section reading:
      stats-collection: 1
    (note: this is for development only, as it assumes you only have one server - in production environments, you will usually list it in the section for a specific server)

Displaying statistics

(note: as of March 28 there is an uncommitted patch on bug 124 [[Routing_and_Template_Toolkit|TTizing] the stats display to DW::Controller::SiteStats, views/admin/stats.tt, view/site/stats.tt)