Code tidy up and more documentation.
This commit is contained in:
parent
abf50bd034
commit
0e814583b7
2 changed files with 95 additions and 61 deletions
96
README.md
96
README.md
|
|
@ -26,7 +26,7 @@ At present several useful functions are implemented:
|
|||
|
||||
Return a list of maps, each representing one account from the `gaza.onl` database, with a key `:donations`, whose value is a float representing the total donations received by that account between the dates `start-date` (exclusive) and `end-date` (inclusive). Date arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L108)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L134)
|
||||
|
||||
### active-accounts
|
||||
|
||||
|
|
@ -36,7 +36,7 @@ Return a list of maps, each representing one account from the `gaza.onl` databas
|
|||
|
||||
Return, as an integer, the number of accounts believed to have been active on this `date`, which should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L97)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L123)
|
||||
|
||||
### average-donations-received-by-month
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ Return, as an integer, the number of accounts believed to have been active on th
|
|||
|
||||
Return data as a map keyed by the date of the first of each month between `start-date` and `end-date`; the values are maps with keys `:average`, `:count` and `:total`. Arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L172)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L194)
|
||||
|
||||
### chart-accounts-with-donations-by-months-in-range
|
||||
|
||||
|
|
@ -56,9 +56,7 @@ Return data as a map keyed by the date of the first of each month between `start
|
|||
|
||||
Return a chart with one line for each calendar month covering from `start-date` (inclusive) to `end-date` (inclusive). Arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`, showing donations as for `chart-donations-by-accounts-in-date-range`, q.v.
|
||||
|
||||
TODO: There are currently two problems with this at present: chart legends are not displayed in sequential order; and there appears to be occasional spurious data (but I don’t know why).
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L142)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L168)
|
||||
|
||||
### chart-average-donations-received-by-month
|
||||
|
||||
|
|
@ -68,7 +66,7 @@ TODO: There are currently two problems with this at present: chart legends are
|
|||
|
||||
Return a chart comparing the average donations received per account with the total number of active accounts per month between `start-date` and `end-date`. Arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L194)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L216)
|
||||
|
||||
### chart-donations-as-percentage-in-date-range
|
||||
|
||||
|
|
@ -78,7 +76,7 @@ Return a chart comparing the average donations received per account with the tot
|
|||
|
||||
Return a chart of donations received by individual accounts between `start-date` (exclusive) and `end-date` (inclusive). Arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`. Accounts are identified in the chart only by rank order.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L129)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L155)
|
||||
|
||||
### chart-donations-by-accounts-in-date-range
|
||||
|
||||
|
|
@ -88,7 +86,7 @@ Return a chart of donations received by individual accounts between `start-date`
|
|||
|
||||
Return a chart of donations received by individual accounts between `start-date` (exclusive) and `end-date` (inclusive). Arguments should be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`. Accounts are identified in the chart only by rank order.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L119)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L145)
|
||||
|
||||
### chart-donations-by-weeks
|
||||
|
||||
|
|
@ -98,7 +96,7 @@ Return a chart of donations received by individual accounts between `start-date`
|
|||
|
||||
Create and return an XY chart of data on field represented by `key` of donations made between `start-date` and `end-date`. Arguments may be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L57)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L83)
|
||||
|
||||
### chart-number-donations-by-weeks
|
||||
|
||||
|
|
@ -108,7 +106,7 @@ Create and return an XY chart of data on field represented by `key` of donations
|
|||
|
||||
Create and return an XY chart of data on number of donations made between `start-date` and `end-date`. Arguments may be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L74)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L100)
|
||||
|
||||
### chart-total-donations-by-weeks
|
||||
|
||||
|
|
@ -118,43 +116,19 @@ Create and return an XY chart of data on number of donations made between `start
|
|||
|
||||
Create and return an XY chart of data on total donations made between `start-date` and `end-date`. Arguments may be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L67)
|
||||
|
||||
### date?
|
||||
|
||||
#### macro
|
||||
|
||||
```
|
||||
(date? x)
|
||||
```
|
||||
|
||||
**TODO**: write docs
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L15)
|
||||
|
||||
### datetime->date
|
||||
|
||||
#### macro
|
||||
|
||||
```
|
||||
(datetime->date x)
|
||||
```
|
||||
|
||||
**TODO**: write docs
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L52)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L93)
|
||||
|
||||
### db
|
||||
|
||||
**TODO**: write docs
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L13)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L19)
|
||||
|
||||
### db-spec
|
||||
|
||||
**TODO**: write docs
|
||||
Database specification in a format which can be used with either JDBC or Korma.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L9)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L12)
|
||||
|
||||
### enhance-account-with-donations
|
||||
|
||||
|
|
@ -164,7 +138,31 @@ Create and return an XY chart of data on total donations made between `start-dat
|
|||
|
||||
Take this `account_map`, which must at least have a valid value for the key `:campaign_url` and return a similar map including the key `:donations`, whose value is a float representing the total donations received by that campaign URL between the dates `sd` (exclusive) and `ed` (inclusive). Date arguments should be supplied as `java.time.LocalDate` objects.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L81)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L107)
|
||||
|
||||
### local-date->date
|
||||
|
||||
#### macro
|
||||
|
||||
```
|
||||
(local-date->date x)
|
||||
```
|
||||
|
||||
Construct and return a new `java.util.Date` object from a `java.time.LocalDate` object. As usual, doing anything with Java time/date stuff is a complete bureaucratic nightmare.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L74)
|
||||
|
||||
### local-date?
|
||||
|
||||
#### macro
|
||||
|
||||
```
|
||||
(local-date? x)
|
||||
```
|
||||
|
||||
Is this `x` an instance of `java.time.LocalDate`?
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L23)
|
||||
|
||||
### normalise-date
|
||||
|
||||
|
|
@ -174,17 +172,17 @@ Take this `account_map`, which must at least have a valid value for the key `:ca
|
|||
(normalise-date date default)
|
||||
```
|
||||
|
||||
**TODO**: write docs
|
||||
More messing around with Java’s bizarre mess of date/time classes. We want our default date representation to be `java.time.LocalDate`, except, of course, when we don’t.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L18)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L28)
|
||||
|
||||
### total-donations-by-weeks
|
||||
|
||||
`(total-donations-by-weeks start-date)``(total-donations-by-weeks start-date end-date)`
|
||||
|
||||
Return a list of pairs [java.time.LocalDate date, float amount](file:///home/simon/workspace/gaza-stats/docs/codox/java.time.LocalDate date, float amount) representing the total of donations made between `start-date` and `end-date`. Arguments may be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`. If `end-date` is not supplied, data for a single week will be returned.
|
||||
Return a list of pairs `[java.time.LocalDate date, float amount]` representing the total of donations made between `start-date` and `end-date`. Arguments may be supplied as `java.time.LocalDate` objects, or as strings in the format `yyyy-mm-dd`. If `end-date` is not supplied, data for a single week will be returned.
|
||||
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L23)
|
||||
[view source](https://git.journeyman.cc/simon/gaza-stats/src/branch/main/src/gaza_stats/core.clj#L45)
|
||||
|
||||
### Viewing charts
|
||||
|
||||
|
|
@ -195,6 +193,16 @@ can then invoke `view` on a chart returned by the above function, for example:
|
|||
(view (chart-total-donations-by-weeks "2024-04-01" "2026-09-20"))
|
||||
```
|
||||
|
||||
### Saving charts as graphics files
|
||||
|
||||
Again, xchart provides facilities for saving charts as graphics files. To do this, in your REPL first invoke `(require '[com.hypirion.clj-xchart :refer [spit]])`. This will, obviously, over-ride `clojure,core/spit`. You can now save a chart as a graphic using, e.g.:
|
||||
|
||||
```clojure
|
||||
(spit (chart-accounts-with-donations-by-months-in-range "2025-01-01" "2025-12-31") "donations-by-months-2025.svg")
|
||||
```
|
||||
|
||||
The graphics format will be taken from the file extension, and at least all of `bmp`, `gif`, `jpg`, `png`, and, most importantly, `svg` are supported. The SVG is properly constructed as a drawing rather than as a raster, although there is no semantic labelling of the nodes or edges, which is disappointing.
|
||||
|
||||
## NOTE: Data
|
||||
|
||||
This code is designed to explore data in a [SQLite](https://www.sqlite.org/) database which it expects to be called `app.db` and find in its `resources` directory. This database is not included in this repository but may be downloaded from [here](https://gaza.onl/app.db).
|
||||
|
|
|
|||
|
|
@ -1,27 +1,49 @@
|
|||
(ns gaza-stats.core
|
||||
"Functions to interrogate the [`gaza.onl`](https://gaza.onl/app.db) database."
|
||||
(:require [clojure.string :refer [capitalize]]
|
||||
[cljc.java-time.local-date :as ld]
|
||||
[com.hypirion.clj-xchart :refer [xy-chart]]
|
||||
[korma.core :refer [as-sql fields join select where]]
|
||||
[korma.db :refer [defdb sqlite3]]))
|
||||
[korma.db :refer [defdb sqlite3]])
|
||||
(:import [java.util Date]
|
||||
[java.time LocalDate ZoneId ZoneOffset]))
|
||||
|
||||
;; create db
|
||||
(def db-spec {:classname "org.sqlite.JDBC"
|
||||
:subprotocol "sqlite"
|
||||
:subname "resources/app.db"})
|
||||
(def db-spec
|
||||
"Database specification in a format which can be used with either JDBC or
|
||||
Korma."
|
||||
{:classname "org.sqlite.JDBC"
|
||||
:subprotocol "sqlite"
|
||||
:subname "resources/app.db"})
|
||||
|
||||
(defdb db (sqlite3 db-spec))
|
||||
(defdb db
|
||||
;; Korma-specific database handle.
|
||||
(sqlite3 db-spec))
|
||||
|
||||
(defmacro date? [x]
|
||||
`(instance? java.time.LocalDate ~x))
|
||||
(defmacro local-date?
|
||||
"Is this `x` an instance of `java.time.LocalDate`?"
|
||||
[x]
|
||||
`(instance? LocalDate ~x))
|
||||
|
||||
(defmacro normalise-date [date default]
|
||||
`(cond (date? ~date) ~date
|
||||
(defmacro normalise-date
|
||||
"More messing around with Java's bizarre mess of date/time classes. We want
|
||||
our default date representation to be `java.time.LocalDate`, except, of
|
||||
course, when we don't."
|
||||
[date default]
|
||||
`(cond (local-date? ~date) ~date
|
||||
(string? ~date) (ld/parse ~date)
|
||||
:else (ld/parse ~default)))
|
||||
;; this one should catch java.sql.Date as well, since it is a subclass
|
||||
(instance? Date ~date) (.toLocalDate
|
||||
(.atZone (.toInstant ~date)
|
||||
(ZoneId/systemDefault)))
|
||||
(nil? ~date) (ld/now)
|
||||
;; if I'd written this as a fn, I'd just recurse,,, might be worth it.
|
||||
:else (if (local-date? ~default)
|
||||
~default
|
||||
(ld/parse (str ~default)))))
|
||||
|
||||
(defn total-donations-by-weeks
|
||||
"Return a list of pairs [java.time.LocalDate date, float amount] representing
|
||||
"Return a list of pairs `[java.time.LocalDate date, float amount]` representing
|
||||
the total of donations made between `start-date` and `end-date`. Arguments may
|
||||
be supplied as `java.time.LocalDate` objects, or as strings in the format
|
||||
`yyyy-mm-dd`. If `end-date` is not supplied, data for a single week will be
|
||||
|
|
@ -49,10 +71,14 @@
|
|||
(if (ld/is-before ed i) (reverse r')
|
||||
(recur i (ld/plus-days i 7) r')))))))
|
||||
|
||||
(defmacro datetime->date [x]
|
||||
`(java.util.Date/from
|
||||
(defmacro local-date->date
|
||||
"Construct and return a new `java.util.Date` object from a
|
||||
`java.time.LocalDate` object. As usual, doing anything with Java time/date
|
||||
stuff is a complete bureaucratic nightmare."
|
||||
[x]
|
||||
`(Date/from
|
||||
(.toInstant (.atStartOfDay ~x)
|
||||
(java.time.ZoneOffset/ofHours 0))))
|
||||
(ZoneOffset/ofHours 0))))
|
||||
|
||||
(defn chart-donations-by-weeks
|
||||
"Create and return an XY chart of data on field represented by `key` of
|
||||
|
|
@ -61,7 +87,7 @@
|
|||
[start-date end-date key]
|
||||
(let [data (total-donations-by-weeks start-date end-date)]
|
||||
(xy-chart {(format "%s Donations" (capitalize (name key)))
|
||||
[(map #(datetime->date (first %)) data)
|
||||
[(map #(local-date->date (first %)) data)
|
||||
(map #(:n (nth % 1)) data)]})))
|
||||
|
||||
(defn chart-total-donations-by-weeks
|
||||
|
|
@ -195,6 +221,6 @@
|
|||
[start-date end-date]
|
||||
(let [data (average-donations-received-by-month start-date end-date)]
|
||||
(xy-chart {(format "Average donations\nper active account\nbetween\n%s and\n%s" start-date end-date)
|
||||
[(map #(datetime->date %)(keys data)) (map :average (vals data))]
|
||||
[(map #(local-date->date %) (keys data)) (map :average (vals data))]
|
||||
(format "Active accounts\nbetween\n%s and\n%s" start-date end-date)
|
||||
[(map #(datetime->date %)(keys data)) (map :count (vals data))]})))
|
||||
[(map #(local-date->date %) (keys data)) (map :count (vals data))]})))
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue