Genuinely pluggable/configurable formatter extensions!

resources/public/content/Introduction.md

@@ -11,27 +11,19 @@ Smeagol is now a fully working small Wiki engine, and meets my own immediate nee
 ## Markup syntax
 Smeagol uses the Markdown format as provided by [markdown-clj](https://github.com/yogthos/markdown-clj), with the addition that anything enclosed in double square brackets, \[\[like this\]\], will be treated as a link into the wiki itself.
 
-## Security and authentication
+### Pluggable extensible markup
-Security is now greatly improved. There is a file called *passwd* in the *resources* directory, which contains a clojure map which maps usernames to maps with plain-text passwords and emails thus:
 
-    {:admin {:password "admin" :email "admin@localhost" :admin true}
+A system of pluggable, extensible formatters is supported. In normal markdown, code blocks may be delimited by three backticks at start and end, and often the syntax of the code can be indicated by a token immediately following the opening three backticks. This has been extended to allow custom formatters to be provided for such code blocks. Two example formatters are provided:
-     :adam {:password "secret" :email "adam@localhost"}}
 
-that is to say, the username is a keyword and the corresponding password is a string. However, since version 0.5.0, users can now change their own passwords, and when the user changes their password their new password is encrypted using the [scrypt](http://www.tarsnap.com/scrypt.html) one-way encryption scheme. The password file is now no longer either in the *resources/public* directory so cannot be downloaded through the browser, nor in the git archive to which the Wiki content is stored, so that even if that git archive is remotely clonable an attacker cannot get the password file that way.
+#### The Vega formatter
 
-## Images
+Inspired by [visdown](http://visdown.amitkaps.com/) and [vega-lite](https://vega.github.io/vega-lite/docs/), the Vega formatter allows you to embed vega data visualisations into Smeagol pages. The graph description should start with a line comprising three back-ticks and then the word '`vega`', and end with a line comprising just three backticks.
-You can (if you're logged in) upload files, including images, using the **Upload a file** link on the top menu bar. You can link to an uploaded image, or other images already available on the web, like this:
 
-![Smeagol](http://vignette3.wikia.nocookie.net/lotr/images/e/e1/Gollum_Render.png/revision/latest?cb=20141218075509)
+Here's an example cribbed in its entirety from [here](http://visdown.amitkaps.com/london):
 
-## Now with data visualisation
+##### Flight punctuality at London airports
 
-Inspired by [visdown](http://visdown.amitkaps.com/) and [vega-lite](https://vega.github.io/vega-lite/docs/), you can now embed visualisations into Smeagol pages, like this:
+```vega
-
-### Flight punctuality at London airports
-
-Example cribbed in its entirety from [here](http://visdown.amitkaps.com/london):
-```vis
 data:
   url: "data/london.csv"
 transform:
@@ -57,14 +49,13 @@ Note that this visualisation will not be rendered in the GitHub wiki, as it does
 
 ![Example visualisation](https://github.com/simon-brooke/smeagol/blob/develop/resources/public/data/london.png?raw=true)
 
-## Now with embedded graphs
+#### The Mermaid formatter
 
-Graphs can now be embedded in a page using the [Mermaid](http://knsv.github.io/mermaid/index.html) graph description language. The graph description should start with a line comprising three back-ticks and then the
+Graphs can now be embedded in a page using the [Mermaid](http://knsv.github.io/mermaid/index.html) graph description language. The graph description should start with a line comprising three back-ticks and then the word `mermaid`, and end with a line comprising just three backticks.
-word 'mermaid', and end with a line comprising just three backticks.
 
 Here's an example culled from the Mermaid documentation.
 
-### GANTT Chart
+##### GANTT Chart
 
 ```mermaid
 gantt
@@ -84,6 +75,27 @@ gantt
         Add to mermaid                      :1d
 ```
 
+To add your own formatter, compile it into a jar file which is on the classpath - it does *not* have to be part of the Smeagol project directly, and then edit the value of the key `:formatters` in the file `config.edn`; whose standard definition is:
+
+     :formatters        {"vega"     smeagol.formatting/process-vega
+                         "vis"      smeagol.formatting/process-vega
+                         "mermaid"  smeagol.formatting/process-mermaid}
+
+The added key should be the word which will follow the opening three backticks of your code block, and the value of that key should be a symbol which evaluates to a function which can format the code block as required.
+
+## Security and authentication
+Security is now greatly improved. There is a file called *passwd* in the *resources* directory, which contains a clojure map which maps usernames to maps with plain-text passwords and emails thus:
+
+    {:admin {:password "admin" :email "admin@localhost" :admin true}
+     :adam {:password "secret" :email "adam@localhost"}}
+
+that is to say, the username is a keyword and the corresponding password is a string. However, since version 0.5.0, users can now change their own passwords, and when the user changes their password their new password is encrypted using the [scrypt](http://www.tarsnap.com/scrypt.html) one-way encryption scheme. The password file is now no longer either in the *resources/public* directory so cannot be downloaded through the browser, nor in the git archive to which the Wiki content is stored, so that even if that git archive is remotely clonable an attacker cannot get the password file that way.
+
+## Images
+You can (if you're logged in) upload files, including images, using the **Upload a file** link on the top menu bar. You can link to an uploaded image, or other images already available on the web, like this:
+
+![Smeagol](http://vignette3.wikia.nocookie.net/lotr/images/e/e1/Gollum_Render.png/revision/latest?cb=20141218075509)
+
 ## Advertisement
 If you like what you see here, I am available for work on open source Clojure projects.
 

resources/public/content/stylesheet.css

@@ -240,6 +240,11 @@ th {
   font-size: 66%;
 }
 
+/* Add space between Vega-Embed links  */
+.vega-actions a {
+  margin-right: 5px;
+}
+
 .warn {
   color: maroon;
 }

resources/templates/wiki.html

@@ -1,23 +1,14 @@
 {% extends "templates/base.html" %}
 
 {% block extra-headers %}
+  {% style "vendor/mermaid/dist/mermaid.css" %}
   <!-- there's at the time of writing (20170731) a problem with the dependencies of the Bower
        package for vega-embed, so we're currently not installing either it or Vega locally.
        TODO: fix -->
   <script src="https://cdnjs.cloudflare.com/ajax/libs/vega/3.0.0-rc2/vega.js"></script>
-  {% script "/vendor/vega-lite/build/vega-lite.js" %}
+  {% script "vendor/vega-lite/build/vega-lite.js" %}
   <script src="https://cdnjs.cloudflare.com/ajax/libs/vega-embed/3.0.0-beta.19/vega-embed.js"></script>
-  <style media="screen">
-    /* Add space between Vega-Embed links  */
-    .vega-actions a {
-      margin-right: 5px;
-    }
-  </style>
 
-  <!-- Similarly, mermaid 7.0.3 isn't installing cleanly for me from Bower. I'm really not sure whether
-       the problem is me getting bower wrong, of these libraries not being well packaged.
-       mermaid 6.0.0 does install, so that's what I'm using. -->
-  {% style "vendor/mermaid/dist/mermaid.css" %}
   {% script "vendor/mermaid/dist/mermaid.js" %}
 {% endblock %}
 

src/smeagol/configuration.clj

@@ -0,0 +1,46 @@
+(ns ^{:doc "Read and make available configuration."
+      :author "Simon Brooke"}
+  smeagol.configuration
+  (:require [noir.io :as io]))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;;;;
+;;;; Smeagol: a very simple Wiki engine.
+;;;;
+;;;; This program is free software; you can redistribute it and/or
+;;;; modify it under the terms of the GNU General Public License
+;;;; as published by the Free Software Foundation; either version 2
+;;;; of the License, or (at your option) any later version.
+;;;;
+;;;; This program is distributed in the hope that it will be useful,
+;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;;; GNU General Public License for more details.
+;;;;
+;;;; You should have received a copy of the GNU General Public License
+;;;; along with this program; if not, write to the Free Software
+;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301,
+;;;; USA.
+;;;;
+;;;; Copyright (C) 2017 Simon Brooke
+;;;;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;;;;
+;;;; Right, doing the data visualisation thing is tricky. Doing it in the
+;;;; pipeline doesn't work, because the md-to-html-string filter messes up
+;;;; both YAML and JSON notation. So we need to extract the visualisation
+;;;; fragments from the Markdown text and replace them with tokens we will
+;;;; recognise afterwards, perform md-to-html-string, and then replace our
+;;;; tokens with the transformed visualisation specification.
+;;;;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+
+(def config-file-path
+  "The relative path to the config file."
+  (str (io/resource-path) "../config.edn"))
+
+
+(def config
+  "The actual configuration, as a map."
+  (read-string (slurp config-file-path)))

src/smeagol/formatting.clj

@@ -10,7 +10,7 @@
             [markdown.core :as md]
             [taoensso.timbre :as timbre]
             [smeagol.authenticate :as auth]
-            ))
+            [smeagol.configuration :refer [config]]))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;;;;
@@ -93,14 +93,6 @@
        "\n</div>"))
 
 
-;; TODO: This isn't (yet) exactly what I want. The formatters ought to be configurable
-;; without editing the Smeagol code directly. But it's a long way in the right direction.
-(def ^:dynamic *formatters*
-  {"vega" process-vega
-   "vis" process-vega
-   "mermaid" process-mermaid})
-
-
 (defn get-first-token
   "Return the first space-separated token of this `string`."
   [^String string]
@@ -163,7 +155,7 @@
   ([index result fragments processed]
    (let [fragment (first fragments)
          first-token (get-first-token fragment)
-         formatter (*formatters* first-token)]
+         formatter (eval ((:formatters config) first-token))]
      (cond
       (empty? fragments)
       (assoc result :text

src/smeagol/routes/wiki.clj

@@ -115,7 +115,6 @@
 (defn wiki-page
   "Render the markdown page specified in this `request`, if any. If none found, redirect to edit-page"
   [request]
-  (timbre/debug (str "Request map: " request))
   (let [params (keywordize-keys (:params request))
         page (or (:page params) (util/get-message :default-page-title request))
         file-name (str "/content/" page ".md")

src/smeagol/util.clj

@@ -6,6 +6,7 @@
             [scot.weft.i18n.core :as i18n]
             [taoensso.timbre :as timbre]
             [smeagol.authenticate :as auth]
+            [smeagol.configuration :refer [config]]
             [smeagol.formatting :refer [md->html]]))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@@ -32,13 +33,6 @@
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 
-;; the relative path to the config file.
-(def config-file-path (str (io/resource-path) "../config.edn"))
-
-
-(def config (read-string (slurp config-file-path)))
-
-
 (defn standard-params
   "Return a map of standard parameters to pass to the template renderer."
   [request]
@@ -64,6 +58,7 @@
 
 (def get-messages (memoize raw-get-messages))
 
+
 (defn get-message
   [message-key request]
   (let [messages (get-messages request)]