group-by: transient-vector buckets + fix nil keys in transient maps (#155)
The map build already used a transient map, but each bucket was rebuilt with
a persistent (conj (get ret k []) x) per element — an O(log n) trie path
rebuild + alloc each. A coarse grouping (few large buckets) was bound on that
conj, not the map build. Buckets are now native arrays (transient vectors,
O(1) push) frozen once; distinct keys are tracked in a side vector so the
buckets freeze in place with no second map rebuild. A bucket's first element
stays a cheap persistent [x] and only promotes to a transient on the second,
so an all-singletons grouping pays no transient alloc.
coarse (10/100 buckets, 50k): ~313ms -> ~125ms (~2.5x)
2 buckets (50k): ~322ms -> ~129ms (~2.5x)
all-unique (50k): ~949ms -> ~892ms (no regression)
Surfaced a latent bug: canon-key returns nil for a nil key and Janet tables
drop a nil key, so the canon-keyed transient map silently lost a nil-key
entry — group-by/frequencies/assoc!/into{} dropped the whole nil bucket
((group-by identity [nil nil 1]) gave {1 [1]}, not {nil [nil nil], 1 [1]}).
Route nil through a sentinel (tbl-key) at the transient-map keying sites;
persistent!/count/dissoc! work unchanged since the real [nil v] pair is kept
as the stored value, and phm already has its own has-nil slot. The transient
set has the analogous bug (needs phs nil support) — filed separately.
Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
parent
1873736045
commit
2668a76837
5 changed files with 56 additions and 9 deletions
|
|
@ -206,5 +206,21 @@
|
|||
["dissoc after bulk" "nil" "(get (dissoc (into {} (map (fn [i] [i i]) (range 100))) 50) 50)"]
|
||||
["frequencies count" "3" "(get (frequencies [1 2 2 1 2 1]) 1)"]
|
||||
["frequencies coll-key" "2" "(get (frequencies [[1 2] [1 2] [3 4]]) [1 2])"]
|
||||
# nil is a legal map key, but a Janet table drops a nil key — the transient
|
||||
# map (canon-keyed table) used to silently lose the nil-key entry, so
|
||||
# group-by/frequencies/assoc dropped the whole nil bucket. tbl-key routes nil
|
||||
# to a sentinel; phm keeps its own has-nil slot.
|
||||
["frequencies nil key" "2" "(get (frequencies [nil nil 1]) nil)"]
|
||||
["group-by nil key" "[nil nil]" "(get (group-by identity [nil nil 1]) nil)"]
|
||||
["group-by nil count" "2" "(count (group-by identity [nil nil 1]))"]
|
||||
["transient nil key" ":x" "(let [t (transient {})] (assoc! t nil :x) (get (persistent! t) nil))"]
|
||||
["transient nil get" "true" "(let [t (transient {})] (assoc! t nil :x) (contains? t nil))"]
|
||||
["transient nil dissoc" ":gone" "(let [t (transient {})] (assoc! t nil :x) (dissoc! t nil) (get (persistent! t) nil :gone))"]
|
||||
# group-by buckets are built on transient vectors (O(1) push) and frozen once,
|
||||
# rather than an O(log n) persistent conj per element. Result is identical:
|
||||
# bucket contents and order match the persistent build across a coarse
|
||||
# grouping (few large buckets — the case bound on the per-bucket conj).
|
||||
["group-by bucket" "[1 3 5]" "(get (group-by odd? (range 1 6)) true)"]
|
||||
["group-by big bucket" "true" "(= (group-by even? (range 200)) {true (vec (filter even? (range 200))) false (vec (filter odd? (range 200)))})"]
|
||||
["group-by order" "[0 3 6 9]" "(get (group-by (fn [x] (mod x 3)) (range 10)) 0)"]
|
||||
["hash-map bulk = incr" "true" "(= (apply hash-map (mapcat (fn [i] [i i]) (range 50))) (reduce (fn [m i] (assoc m i i)) {} (range 50)))"])
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue