Small amount more tidyup

`load-microworld-edn` now tested with a large data set.

.gitignore

@@ -33,3 +33,5 @@ small_hill.html
 .settings/org.eclipse.jdt.apt.core.prefs
 .settings/org.eclipse.jdt.core.prefs
 .settings/org.eclipse.m2e.core.prefs
+
+resources/superstructure-raw-20240410.edn

README.md

@@ -8,6 +8,96 @@ This library is written in support of work on
 [The Great Game](https://simon-brooke.github.io/the-great-game/codox/Pathmaking.html), but is
 separate because it may be of some use in other settings.
 
+## Explanation
+
+*I'm writing this becaue, having returned to this project after four years of 
+being distracted by other things, I couldn't even remember what it was for, 
+let alone how to use it.*
+
+### 'STL files'
+
+'STL' is not an acronym but an abbreviation; an abreviation of 
+'[stereolithography](https://en.wikipedia.org/wiki/Stereolithography)'. Even 
+that name is completely misleading, since there is absolutely no sense in
+which this is printing with stone.
+
+What an STL file is is in fact a representation of a mesh, originally intended 
+for use in 3d printing, and therefore taken to be the bounding mesh of a solid.
+This makes it a better format for representing landforms for games than a
+simple heightmap, since heightmaps cannot represent things like overhangs and 
+caves.
+
+STL files come in two subformats: 'ASCII' (which this library can currently 
+read and write), and 'Binary' (which this library can read but not yet write).
+
+STL files are commonly used in game development and can be imported by all of
+
+* [Blender](https://docs.blender.org/manual/en/latest/files/import_export/stl.html)
+* [Godot Engine](https://godotengine.org/asset-library/asset/961)
+* [Unity](https://github.com/karl-/pb_Stl) (unofficial)
+* [Unreal Engine](https://github.com/rdeioris/UnrealSTL) (unofficial)
+
+Additionally, a number of programs exist to convert STL files into other 3D
+model file formats.
+
+### 'Superstructure'
+
+I can't remember if 'superstructure' is a name which I found in reading the
+literature on STL files, or whether it's one I came up with for my own 
+purposes, but essentially in my usage of it:
+
+1. [MicroWorld](https://github.com/simon-brooke/mw-engine) is a two dimensional 
+cellular automaton, which I wrote, and which can be used (and which I use) to 
+model ecological processes up to and including human settlement;
+2. As a two dimensional cellular automaton, it is necessarily essentially a
+flat mesh, but I essentially drape that over a 
+[heightmap](https://github.com/simon-brooke/mw-engine/blob/master/src/mw_engine/heightmap.clj)
+to give a sort of notional two-and-a-half dimensional representation;
+3. As such MicroWorld is a moderately useful tool to transform an imagined land
+form into a vegetated and populated landscape, with biomes and human habitation
+distributed naturalistically.
+
+A 'superstructure', then, in the terms I'm using it, is an intermediary 
+representation between a heightmap and a stereolithography model, which carries
+data not only about shape, but about biomes, settlement, traversibility and so
+on. Thus it is a surface across which routes can efficiently be planned.
+
+A superstructure can currently be constructed from a MicroWorld world, read in
+from an [Extensible Data Notation](https://github.com/edn-format/edn). There 
+isn't currently a function to construct a superstructure from an in-memory
+MicroWorld world, because MicroWorld, when working with models of the size I'm 
+using, is hugely memory hungry; but there clearly could be and that is an
+addition you should expect.
+
+A superstructure cannot currently be constructed directly from a heightmap, but
+that would also be a trivial enhancement and is planned.
+
+```clojure
+(use 'cc.journeyman.walkmap.microworld :reload)
+(require '[cc.journeyman.walkmap.vertex :refer [vertex]])
+(load-microworld-edn 
+    "../the-great-game/resources/test/galloway-populated-20240407.edn")
+```
+or
+```clojure
+(use 'cc.journeyman.walkmap.microworld :reload)
+(require '[cc.journeyman.walkmap.vertex :refer [vertex]])
+(load-microworld-edn 
+    "../the-great-game/resources/test/galloway-populated-20240407.edn" :mw {} 
+    (vertex 1000 1000 10))
+```
+
+Note that this is a lot of processing and even on a fast machine is going to 
+take significant walkclock time. You cannot reasonably do it, except for 
+utterly trivial times, during game initialisation; instead you need to do it
+in the 'baking' phase of game development, and consider the STL file as the
+resoource to be shipped with the game.
+
+However, although it is time-expensive, it is not, unlike MicroWorld, either
+memory or processor hungry. This will not blow up your machine.
+
+A superstructure can be written to an ASCII STL file.
+
 ## Usage
 
 What works:

src/cc/journeyman/walkmap/microworld.clj

@@ -10,7 +10,7 @@
             [clojure.edn :as edn :only [read]]
             [clojure.java.io :refer [reader]]
             [taoensso.timbre :refer [error]])
-  (:import [clojure.lang Keyword Map]
+  (:import [clojure.lang Keyword IPersistentMap]
            [java.io PushbackReader]))
 
 (defn cell->polygon
@@ -21,20 +21,20 @@
    (cell->polygon cell (vertex 1 1 1)))
   ([cell scale-vector]
    (tag
-     (assoc
+    (assoc
-       (merge
+     (merge
-         cell
+      cell
-         (let [w (* (:x cell) (:x (check-vertex scale-vector)))
+      (let [w (* (:x cell) (:x (check-vertex scale-vector)))
-               s (* (:y cell) (:y scale-vector))
+            s (* (:y cell) (:y scale-vector))
-               e (+ w (:x scale-vector))
+            e (+ w (:x scale-vector))
-               n (+ s (:y scale-vector))
+            n (+ s (:y scale-vector))
-               z (* (:altitude cell) (:z scale-vector))]
+            z (* (:altitude cell) (:z scale-vector))]
-         (rectangle
+        (rectangle
-           (vertex s w z)
+         (vertex s w z)
-           (vertex n e z))))
+         (vertex n e z))))
-       :walkmap.id/id
+     :walkmap.id/id
-       (keyword (gensym "mw-cell")))
+     (keyword (gensym "mw-cell")))
-     (:state cell))))
+    (:state cell))))
 
 (defn load-microworld-edn
   "While it would be possible to call MicroWorld functions directly from
@@ -45,29 +45,29 @@
   ([^String filename]
    (load-microworld-edn filename :mw))
   ([^String filename ^Keyword map-kind]
-   (when-not
-     (keyword? map-kind)
-     (throw (IllegalArgumentException.
-              (truncate
-                (format "Must be a keyword: %s." (or map-kind "nil")) 80))))
    (load-microworld-edn filename map-kind nil))
-  ([^String filename ^Keyword mapkind ^Map superstucture]
+  ([^String filename ^Keyword mapkind ^IPersistentMap superstucture]
    (load-microworld-edn filename mapkind superstucture (vertex 1 1 1)))
-  ([^String filename ^Keyword _map-kind ^Map superstructure ^Map _scale-vertex]
+  ([^String filename ^Keyword map-kind ^IPersistentMap superstructure ^IPersistentMap _scale-vertex]
+   (when-not
+    (keyword? map-kind)
+     (throw (IllegalArgumentException.
+             (truncate
+              (format "Must be a keyword: %s." (or map-kind "nil")) 80))))
    (let [mw (try
               (with-open [r (reader filename)]
                 (edn/read (PushbackReader. r)))
               (catch RuntimeException e
                 (error "Error parsing edn file '%s': %s\n"
-                         filename (.getMessage e))))
+                       filename (.getMessage e))))
          polys (reduce
-                 concat
+                concat
-                 (map (fn [row] (map cell->polygon row)) mw))]
+                (map (fn [row] (map cell->polygon row)) mw))]
      (if (map? superstructure)
        (reduce
-         #(store %2 %1)
+        #(store %2 %1)
-         superstructure
+        superstructure
-         polys)
+        polys)
        polys))))
 
 

src/cc/journeyman/walkmap/superstructure.clj

@@ -7,7 +7,7 @@
             ;; [cc.journeyman.walkmap.polygon :as q]
             [cc.journeyman.walkmap.utils :as u]
             [cc.journeyman.walkmap.vertex :as v])
-  (:import [clojure.lang IFn Map]))
+  (:import [clojure.lang IFn IPersistentMap]))
 
 ;; TODO: Think about reification/dereification. How can we cull a polygon, if
 ;; some vertices still index it? I *think* that what's needed is that when
@@ -54,7 +54,7 @@
   2. `o` is not a map;
   3. `o` does not have a value for the key `:walkmap.id/id`;
   4. `v` is not a vertex."
-  [^Map s ^Map o ^Map v]
+  [^IPersistentMap s ^IPersistentMap o ^IPersistentMap v]
   (if-not (v/vertex? o)
     (if (:walkmap.id/id o)
       (if (v/vertex? v)
@@ -162,9 +162,9 @@
 
   1. `s` is not a map;
   2. `o` is not a recognisable walkmap object"
-  ([^Map o]
+  ([^IPersistentMap o]
    (store o {}))
-  ([^Map o ^Map s]
+  ([^IPersistentMap o ^IPersistentMap s]
    (when-not (:walkmap.id/id o)
      (throw
       (IllegalArgumentException.
@@ -189,7 +189,7 @@
   have properties which will be denormalised by `store`, and therefore do not
   have to restored with `retrieve`. If properties are added to vertices
   whose values are objects, then this will have to be rewritten."
-  ([^Map s ^Map minv ^Map maxv]
+  ([^IPersistentMap s ^IPersistentMap minv ^IPersistentMap maxv]
    (search-vertices s minv maxv false))
   ([s minv maxv d2?]
    (let [minv' (if d2? (assoc minv :z Double/NEGATIVE_INFINITY) minv)
@@ -206,9 +206,9 @@
 
   WARNING: currently only returns objects which have a defined `:centre`
   (but most of the significant objects we have do)."
-  ([^Map s ^Map target ^Number radius]
+  ([^IPersistentMap s ^IPersistentMap target ^Number radius]
    (nearest s target :centre radius))
-  ([^Map s ^Map target ^IFn filter-fn ^Number radius]
+  ([^IPersistentMap s ^IPersistentMap target ^IFn filter-fn ^Number radius]
    (let [minv (v/vertex
                (- (:x (v/check-vertex target)) radius)
                (- (:y target) radius) (- (or (:z target) 0) radius))
@@ -225,7 +225,7 @@
         filter-fn
         (filter
          :centre
-         (map #(retrieve % s)
+         (pmap #(retrieve % s)
                ;; for each vertex id in vids, get the objects associated with that id
                ;; in the vertex index as a single flat list
               (reduce
@@ -243,11 +243,11 @@
 (defn touching
   "Return a sequence of all objects in superstructure `s` which are
   indexed as touching the vertex `v`."
-  ([^Map vertex ^Map s]
+  ([^IPersistentMap vertex ^IPersistentMap s]
    (map
     #(retrieve % s)
     (set (-> s :vertex-index (:walkmap.id/id (v/check-vertex vertex)) keys))))
-  ([^Map vertex ^IFn filter-fn ^Map s]
+  ([^IPersistentMap vertex ^IFn filter-fn ^IPersistentMap s]
    (filter
     filter-fn
     (touching vertex s))))
@@ -256,9 +256,9 @@
   "Return a sequence of all those objects in superstructure `s` which share
   at least one vertex with `target`, and which are matched by `filter-fn`
   if supplied."
-  ([^Map target ^Map s]
+  ([^IPersistentMap target ^IPersistentMap s]
    (neighbours target identity s))
-  ([^Map target ^IFn filter-fn ^Map s]
+  ([^IPersistentMap target ^IFn filter-fn ^IPersistentMap s]
    (remove
     #(= target %)
     (reduce
@@ -271,7 +271,7 @@
   "Return a sequence of the ids all those objects in superstructure `s` which
   share at least one vertex with `target`, and which are matched by
   `filter-fn` if supplied."
-  ([^Map target ^Map s]
+  ([^IPersistentMap target ^IPersistentMap s]
    (neighbour-ids target identity s))
-  ([^Map target ^IFn filter-fn ^Map s]
+  ([^IPersistentMap target ^IFn filter-fn ^IPersistentMap s]
    (map :walkmap.id/id (neighbours target filter-fn s))))

src/cc/journeyman/walkmap/tag.clj

@@ -5,7 +5,7 @@
   (:require [clojure.set :refer [difference union]]
             [taoensso.timbre :as l]
             [cc.journeyman.walkmap.utils :refer [kind-type]])
-  (:import [clojure.lang Map]))
+  (:import [clojure.lang IPersistentMap]))
 
 (defn tagged?
   "True if this `object` is tagged with each of these `tags`. It is an error
@@ -13,7 +13,7 @@
 
   1. `object` is not a map;
   2. any of `tags` is not a keyword."
-  [^Map object & tags]
+  [^IPersistentMap object & tags]
   (let [tags' (flatten tags)]
     (when-not (every? keyword? tags')
       (throw (IllegalArgumentException.
@@ -33,7 +33,7 @@
 
   It's legal to include sequences of keywords in `tags`, so that users can do
   useful things like `(tag obj (map keyword some-strings))`."
-  [^Map object & tags]
+  [^IPersistentMap object & tags]
   (l/debug "Tagging" (kind-type object) "with" tags)
   (let [tags' (flatten tags)]
     (when-not (every? keyword? tags')
@@ -52,7 +52,7 @@
 
   1. `object` is not a map;
   2. any of `tags` is not a keyword or sequence of keywords."
-  [^Map object & tags]
+  [^IPersistentMap object & tags]
   (let [tags' (flatten tags)]
     (when-not (every? keyword? tags')
       (throw (IllegalArgumentException.

src/cc/journeyman/walkmap/vertex.clj

@@ -6,7 +6,8 @@
   (:require [clojure.math.numeric-tower :as m]
             [clojure.string :as s]
             [taoensso.timbre :as l]
-            [cc.journeyman.walkmap.utils :refer [=ish check-kind-type check-kind-type-seq kind-type truncate]]))
+            [cc.journeyman.walkmap.utils :refer 
+             [=ish check-kind-type check-kind-type-seq kind-type truncate]]))
 
 (defn vertex-key
   "Making sure we get the same key everytime we key a vertex with the same