+ statistics: store in separate file

+ GET /control/stats handler

AGHTechDoc.md

@@ -37,6 +37,11 @@ Contents:
 * Services Filter
 	* API: Get blocked services list
 	* API: Set blocked services list
+* Statistics
+	* API: Get statistics data
+	* API: Clear statistics data
+	* API: Set statistics parameters
+	* API: Get statistics parameters
 
 
 ## First startup
@@ -863,3 +868,111 @@ Request:
 Response:
 
 	200 OK
+
+
+## Statistics
+
+Load (main thread):
+. Load data from the last bucket from DB for the current hour
+
+Runtime (DNS worker threads):
+. Update current unit
+
+Runtime (goroutine):
+. Periodically check that current unit should be flushed to file (when the current hour changes)
+ . If so, flush it, allocate a new empty unit
+
+Runtime (HTTP worker threads):
+. To respond to "Get statistics" API request we:
+ . load all units from file
+ . load current unit
+ . process data from all loaded units:
+  . sum up data for "total counters" output values
+  . add value into "per time unit counters" output arrays
+  . aggregate data for "top_" output arrays;  sort in descending order
+
+Unload (main thread):
+. Flush current unit to file
+
+
+### API: Get statistics data
+
+Request:
+
+	GET /control/stats
+
+Response:
+
+	200 OK
+
+	{
+		time_units: hours | days
+
+		// total counters:
+		num_dns_queries: 123
+		num_blocked_filtering: 123
+		num_replaced_safebrowsing: 123
+		num_replaced_safesearch: 123
+		num_replaced_parental: 123
+		avg_processing_time: 123.123
+
+		// per time unit counters
+		dns_queries: [123, ...]
+		blocked_filtering: [123, ...]
+		replaced_parental: [123, ...]
+		replaced_safebrowsing: [123, ...]
+
+		top_queried_domains: [
+			{host: 123},
+			...
+		]
+		top_blocked_domains: [
+			{host: 123},
+			...
+		]
+		top_clients: [
+			{IP: 123},
+			...
+		]
+	}
+
+
+### API: Clear statistics data
+
+Request:
+
+	POST /control/stats_reset
+
+Response:
+
+	200 OK
+
+
+### API: Set statistics parameters
+
+Request:
+
+	POST /control/stats_config
+
+	{
+		"interval": 1 | 7 | 30 | 90
+	}
+
+Response:
+
+	200 OK
+
+
+### API: Get statistics parameters
+
+Request:
+
+	GET /control/stats_info
+
+Response:
+
+	200 OK
+
+	{
+		"interval": 1 | 7 | 30 | 90
+	}