Difference between revisions of "Stats Design"
From Dreamwidth Notes
(Another link) |
|||
| (16 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | {{Update}} | |
| − | + | This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.) | |
| − | + | [[Project_Teams#Site_statistics|Stats project team]] | |
| − | CREATE TABLE statkeylist ( | + | == Table definitions == |
| + | |||
| + | <source lang="sql">CREATE TABLE statkeylist ( | ||
statkeyid int unsigned NOT NULL auto_increment, | statkeyid int unsigned NOT NULL auto_increment, | ||
name varchar(255) default NULL, | name varchar(255) default NULL, | ||
| Line 11: | Line 13: | ||
PRIMARY KEY (statkeyid), | PRIMARY KEY (statkeyid), | ||
UNIQUE KEY (name) | UNIQUE KEY (name) | ||
| − | ) | + | ) |
CREATE TABLE site_stats ( | CREATE TABLE site_stats ( | ||
| Line 23: | Line 25: | ||
-- time (ie, last 5 entries, or last 2 weeks). Do we need an extra index? | -- time (ie, last 5 entries, or last 2 weeks). Do we need an extra index? | ||
INDEX (category_id, key_id, insert_time) | INDEX (category_id, key_id, insert_time) | ||
| − | ) | + | )</source> |
| + | |||
| + | == Storing and retrieving statistics == | ||
| + | |||
| + | [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/DW/StatStore.pm DW::StatStore] | ||
| + | |||
| + | == Statistics items == | ||
| + | |||
| + | * [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/DW/StatData.pm DW::StatData] | ||
| + | * [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/DW/StatData/ DW::StatData::*] | ||
| + | |||
| + | == Collecting statistics == | ||
| + | |||
| + | This works by massaging, aggregating, and or compiling (as appropriate) data from various places and storing the results using DW::StatStore. | ||
| + | |||
| + | 1. [https://github.com/dreamwidth/dw-free/blob/develop/etc/stats-collection.conf etc/stats-collection.conf] should contain something like: | ||
| + | <source lang="text">## collect and store all stats under the accounts category | ||
| + | ## hangs if you don't enclose the asterisk in quotes | ||
| + | # accounts: "*" | ||
| + | ## collect and store only these stats under the accounts category | ||
| + | # accounts: [ identity, personal, redirect ] | ||
| + | accounts: "*" | ||
| + | active: "*" | ||
| + | paid: "*" | ||
| + | </source> | ||
| + | |||
| + | 2. [https://github.com/dreamwidth/dw-free/blob/develop/bin/worker/stats-collection bin/worker/stats-collection] is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in [https://github.com/dreamwidth/dw-free/blob/develop/etc/workers.conf etc/workers.conf]. | ||
| − | + | 3. (only needed if you chose to use [https://github.com/dreamwidth/dw-free/blob/develop/etc/workers.conf etc/workers.conf]): add a line to the "all" section reading: | |
| + | <source lang="text"> stats-collection: 1 | ||
| + | </source>(note: this is for development only, as it assumes you only have one server - in production environments, you will usually list it in the section for a specific server) | ||
| − | + | == Displaying statistics == | |
| − | + | * [https://github.com/dreamwidth/dw-free/blob/develop/views/stats/site.tt views/stats/site.tt] (public stats) | |
| + | * [https://github.com/dreamwidth/dw-free/blob/develop/views/admin/stats.tt views/admin/stats.tt] (restricted stats) | ||
| + | * [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/DW/Controller/SiteStats.pm DW::Controller::SiteStats.pm] (the stats page controller for both public and restricted stats) | ||
| + | * [https://github.com/dreamwidth/dw-free/blob/develop/cgi-bin/DW/Controller/Graphs.pm DW::Controller::Graphs.pm] (the stats graphs controller for both public and restricted stats) | ||
| − | + | [[Category: Development]] | |
Latest revision as of 10:08, 4 August 2014
Needs Update: This page has been found in need of an update. Information may be wrong or outdated!
This is a a loosely organized collection of notes and links for the Stats feature design. (Note: this is for the new stats system only - there's an existing legacy system too.)
Contents
Table definitions
CREATE TABLE statkeylist ( statkeyid INT UNSIGNED NOT NULL AUTO_INCREMENT, name VARCHAR(255) DEFAULT NULL, PRIMARY KEY (statkeyid), UNIQUE KEY (name) ) CREATE TABLE site_stats ( category_id INT UNSIGNED NOT NULL, key_id INT UNSIGNED NOT NULL, insert_time INT UNSIGNED NOT NULL, VALUE INT UNSIGNED NOT NULL, -- FIXME: This is good for retrieving data for a single category+key, but -- maybe not as good if we want all keys for the category, with a limit on -- time (ie, last 5 entries, or last 2 weeks). Do we need an extra index? INDEX (category_id, key_id, insert_time) )
Storing and retrieving statistics
Statistics items
Collecting statistics
This works by massaging, aggregating, and or compiling (as appropriate) data from various places and storing the results using DW::StatStore.
1. etc/stats-collection.conf should contain something like:
## collect and store all stats under the accounts category ## hangs if you don't enclose the asterisk in quotes # accounts: "*" ## collect and store only these stats under the accounts category # accounts: [ identity, personal, redirect ] accounts: "*" active: "*" paid: "*"
2. bin/worker/stats-collection is the utility that handles stats collection. Run at least once per 24 hours, either from your crontab or by listing it in etc/workers.conf.
3. (only needed if you chose to use etc/workers.conf): add a line to the "all" section reading:
stats-collection: 1
Displaying statistics
- views/stats/site.tt (public stats)
- views/admin/stats.tt (restricted stats)
- DW::Controller::SiteStats.pm (the stats page controller for both public and restricted stats)
- DW::Controller::Graphs.pm (the stats graphs controller for both public and restricted stats)
