Difference between revisions of "Stats Design"

From Dreamwidth Notes
Jump to: navigation, search
(Displaying statistics: Track URL for revised patch)
(Update bug links, remove obsolete "we're recruiting")
Line 1: Line 1:
 
This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.)
 
This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.)
  
[http://bugs.dwscoalition.org/show_bug.cgi?id=124 Bug 124]
+
[http://bugs.dwscoalition.org/show_bug.cgi?id=2717 Bug 2717] and its dependent bugs
  
[[Project_Teams#Site_statistics|Stats project team]] (we're recruiting)
+
[[Project_Teams#Site_statistics|Stats project team]]
  
 
== Table definitions ==
 
== Table definitions ==
Line 62: Line 62:
 
* [http://hg.dwscoalition.org/dw-free/file/tip/htdocs/admin/stats.bml htdocs/admin/stats.bml] (restricted stats)
 
* [http://hg.dwscoalition.org/dw-free/file/tip/htdocs/admin/stats.bml htdocs/admin/stats.bml] (restricted stats)
  
(note: as of June 19 there is an [http://bugs.dwscoalition.org/attachment.cgi?id=3716 uncommitted patch] on bug 124 [[Routing_and_Template_Toolkit|TTizing]] the stats display to DW::Controller::SiteStats, views/admin/stats.tt, view/site/stats.tt)
+
(note: as of June 19 there is an [http://bugs.dwscoalition.org/attachment.cgi?id=3716 uncommitted patch] on [http://bugs.dwscoalition.org/show_bug.cgi?id=124 bug 124] [[Routing_and_Template_Toolkit|TTizing]] the stats display to DW::Controller::SiteStats, views/admin/stats.tt, view/site/stats.tt)

Revision as of 18:55, 21 June 2010

This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.)

Bug 2717 and its dependent bugs

Stats project team

Table definitions

CREATE TABLE statkeylist (
    statkeyid INT UNSIGNED NOT NULL AUTO_INCREMENT, 
    name VARCHAR(255) DEFAULT NULL, 
 
    PRIMARY KEY (statkeyid), 
    UNIQUE KEY (name) 
)
 
CREATE TABLE site_stats ( 
    category_id INT UNSIGNED NOT NULL, 
    key_id INT UNSIGNED NOT NULL, 
    insert_time INT UNSIGNED NOT NULL, 
    VALUE INT UNSIGNED NOT NULL, 
 
    -- FIXME: This is good for retrieving data for a single category+key, but 
    -- maybe not as good if we want all keys for the category, with a limit on 
    -- time (ie, last 5 entries, or last 2 weeks). Do we need an extra index? 
    INDEX (category_id, key_id, insert_time) 
)

Storing and retrieving statistics

DW::StatStore

Statistics items

Collecting statistics

This works by massaging, aggregating, and or compiling (as appropriate) data from various places and storing the results using DW::StatStore.

1. etc/stats-collection.conf should contain something like:

## collect and store all stats under the accounts category
## hangs if you don't enclose the asterisk in quotes
# accounts: "*"
## collect and store only these stats under the accounts category
# accounts: [ identity, personal, redirect ]
accounts: "*"
active: "*"
paid: "*"

2. bin/worker/stats-collection is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in etc/workers.conf.

3. (only needed if you chose to use etc/workers.conf): add a line to the "all" section reading:

  stats-collection: 1
(note: this is for development only, as it assumes you only have one server - in production environments, you will usually list it in the section for a specific server)

Displaying statistics

(note: as of June 19 there is an uncommitted patch on bug 124 TTizing the stats display to DW::Controller::SiteStats, views/admin/stats.tt, view/site/stats.tt)