Merge branch 'feature/30' into develop

This commit is contained in:
Simon Brooke 2017-09-09 23:38:29 +01:00
commit b6a2bdd4bc
15 changed files with 306 additions and 168 deletions

View file

@ -1,19 +1,20 @@
(defproject smeagol "0.99.8"
(defproject smeagol "0.99.9-SNAPSHOT"
:description "A simple Git-backed Wiki inspired by Gollum"
:url "https://github.com/simon-brooke/smeagol"
:license {:name "GNU General Public License,version 2.0 or (at your option) any later version"
:url "https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html"}
:dependencies [[clj-jgit "0.8.9"]
:dependencies [[clj-jgit "0.8.10"]
[clj-yaml "0.4.0"]
[com.cemerick/url "0.1.1"]
[com.fzakaria/slf4j-timbre "0.3.7"]
[com.taoensso/encore "2.91.1"]
[com.taoensso/encore "2.92.0"]
[com.cemerick/url "0.1.1"]
[com.taoensso/timbre "4.10.0"]
[com.fzakaria/slf4j-timbre "0.3.7"]
[com.taoensso/tower "3.0.2" :exclusions [com.taoensso/encore]]
[crypto-password "0.2.0"]
[environ "1.1.0"]
[hiccup "1.0.5"]
[im.chit/cronj "1.4.4"]
[lib-noir "0.9.9" :exclusions [org.clojure/tools.reader]]
[markdown-clj "0.9.99" :exclusions [com.keminglabs/cljx]]

View file

@ -6,11 +6,19 @@ To deploy Smeagol as a stand-alone application, either download the jar file for
This will create a jar file in the `target` directory, named `smeagol-`*VERSION*`-standalone.jar`.
Smeagol cannot access either its configuration or its content from the jar file. Consequently you should set up three environment variables:
Smeagol cannot access either its configuration or its content from the jar file, as otherwise they would not be editable. Consequently you should set up three environment variables:
1. **SMEAGOL_CONFIG** should be the full or relative pathname of a Smeagol [[Configuration]] file;
2. **SMEAGOL\_CONTENT\_DIR** should be the full or relative pathname of the directory from which Smeagol should serve content (which may initially be empty, but must be writable by the process which runs Smeagol)'
3. **SMEAGOL_PASSWD** should be the full or relative pathname of a Smeagol Passwd file - see [[Security and authentication]]. This file must contain an entry for at least your initial user, and, if you want to administer users through the user interface, must be writable by the process which runs Smeagol);
1. `SMEAGOL_CONFIG` should be the full or relative pathname of a Smeagol [[Configuration]] file;
2. `SMEAGOL_CONTENT_DIR` should be the full or relative pathname of the directory from which Smeagol should serve content (which may initially be empty, but must be writable by the process which runs Smeagol)'
3. `SMEAGOL_PASSWD` should be the full or relative pathname of a Smeagol Passwd file - see [[Security and authentication]]. This file must contain an entry for at least your initial user, and, if you want to administer users through the user interface, must be writable by the process which runs Smeagol.
**NOTE** that `SMEAGOL_CONTENT_DIR` must contain at least the following files:
1. `_edit-side-bar.md` - the side-bar that should be displayed when editing pages;
2. `_header.md` - the header to be displayed on all pages;
3. `_side-bar.md` - the side-bar that should be displayed when not editing pages.
Standard versions of these files can be found in the [source repository](https://github.com/journeyman-cc/smeagol/tree/master/resources/public/content). All these files should be in markdown format - see [[Extensible Markup]].
You can run the jar file with:

View file

@ -13,7 +13,7 @@ To start a web server for the application during development, run:
This should start a development server, and open a new window or tab in your default browser with the default page of the wiki loaded into it.
## Editing
I generally use [LightTable]() as my `Clojure` editor, but it doesn't really matter what you use; if you run Smeagol as described above, then all changes you make in the code (and save) will instantly be applied to the running system. This makes for a productive development environment.
I generally use [LightTable](http://lighttable.com/) as my `Clojure` editor, but it doesn't really matter what you use; if you run Smeagol as described above, then all changes you make in the code (and save) will instantly be applied to the running system. This makes for a productive development environment.
## Documentation
It is my intention that the code should be sufficiently well documented to be easy to understand. Documentation may be generated from the code by running

View file

@ -1,3 +1,5 @@
The basic format of Smeagol pages is [Markdown](https://daringfireball.net/projects/markdown/); documentation on how to format them is [here](https://daringfireball.net/projects/markdown/syntax). Note that there are a number of slightly different variants of Markdown; the version used by Smeagol does not currently allow tables.
A system of pluggable, extensible formatters is supported. In normal markdown, code blocks may be delimited by three backticks at start and end, and often the syntax of the code can be indicated by a token immediately following the opening three backticks. This has been extended to allow custom formatters to be provided for such code blocks. Two example formatters are provided:
## The Vega formatter

View file

@ -128,7 +128,7 @@
"Return the map of features of this user, if any."
[username]
(if
(and username (> (count (str username)) 0))
(and username (pos? (count (str username))))
((keyword username) (get-users))))
@ -138,7 +138,7 @@
(timbre/info "Trying to add user " username)
(cond
(not (string? username)) (throw (Exception. "Username must be a string."))
(= (count username) 0) (throw (Exception. "Username cannot be zero length"))
(zero? (count username)) (throw (Exception. "Username cannot be zero length"))
true (let [users (get-users)
user ((keyword username) users)
password (if
@ -146,7 +146,7 @@
(password/encrypt newpass))
details {:email email
:admin (if
(and (string? admin) (> (count admin) 0))
(and (string? admin) (pos? (count admin)))
true
false)}
;; if we have a valid password we want to include it in the details to update.

View file

@ -46,4 +46,7 @@
(def config
"The actual configuration, as a map."
(read-string (slurp config-file-path)))
(try
(read-string (slurp config-file-path))
(catch Exception any
(throw (Exception. "Could not load configuration" any)))))

View file

@ -47,16 +47,6 @@
(defn diff2html
"Convert this string, assumed to be in diff format, to HTML."
[^String diff-text]
(apply str
(flatten
(list "<div class='change'>"
(join "\n"
(remove nil?
(map mung-line
;; The first five lines are boilerplate, and
;; uninteresting for now
(drop 5
(split-lines diff-text)))))
"</div>"))))
(clojure.string/join (flatten (list "<div class='change'>" (join "\n" (remove nil? (map mung-line (drop 5 (split-lines diff-text))))) "</div>"))))

View file

@ -115,7 +115,7 @@
corresponding inclusion should be inserted."
[index result fragment fragments processed]
(process-text
(+ index 1)
(inc index)
result
fragments
(cons fragment processed)))
@ -133,18 +133,8 @@
(let
[kw (keyword (str "inclusion-" index))]
(process-text
(+ index 1)
(assoc
result
:inclusions
(assoc
(:inclusions result)
kw
(apply
formatter
(list
(subs fragment (count token))
index))))
(inc index)
(assoc-in result [:inclusions kw] (apply formatter (list (subs fragment (count token)) index)))
(rest fragments)
(cons kw processed))))

View file

@ -1,7 +1,8 @@
(ns ^{:doc "Set up, configure, and clean up after the wiki server."
:author "Simon Brooke"}
smeagol.handler
(:require [compojure.core :refer [defroutes]]
(:require [clojure.java.io :as cjio]
[compojure.core :refer [defroutes]]
[compojure.route :as route]
[cronj.core :as cronj]
[environ.core :refer [env]]
@ -46,23 +47,6 @@
(route/resources "/")
(route/not-found "Not Found"))
(defn init
"init will be called once when
app is deployed as a servlet on
an app server such as Tomcat
put any initialization code here"
[]
(timbre/merge-config!
{:appenders
{:rotor (rotor/rotor-appender
{:path "smeagol.log"
:max-size (* 512 1024)
:backlog 10})}})
(if (env :dev) (parser/cache-off!))
;;start the expired session cleanup job
(cronj/start! session-manager/cleanup-job)
(timbre/info "\n-=[ smeagol started successfully"
(when (env :dev) "using the development profile") "]=-"))
(defn destroy
"destroy will be called when your application
@ -72,28 +56,53 @@
(cronj/shutdown! session-manager/cleanup-job)
(timbre/info "shutdown complete!"))
(defn init
"init will be called once when
app is deployed as a servlet on
an app server such as Tomcat
put any initialization code here"
[]
(try
(timbre/merge-config!
{:appenders
{:rotor (rotor/rotor-appender
{:path "smeagol.log"
:max-size (* 512 1024)
:backlog 10})}})
(cronj/start! session-manager/cleanup-job)
(if (env :dev) (parser/cache-off!))
;;start the expired session cleanup job
(timbre/info "\n-=[ smeagol started successfully"
(when (env :dev) "using the development profile") "]=-")
(catch Exception any
(timbre/error "Failure during startup" any)
(destroy))))
;; timeout sessions after 30 minutes
(def session-defaults
{:timeout (* 60 30)
:timeout-response (redirect "/")})
(defn- mk-defaults
"set to true to enable XSS protection"
[xss-protection?]
(-> site-defaults
(update-in [:session] merge session-defaults)
(assoc-in [:security :anti-forgery] xss-protection?)))
(defn- make-defaults
"set to true to enable XSS protection"
[xss-protection?]
(-> site-defaults
(update-in [:session] merge session-defaults)
(assoc-in [:security :anti-forgery] xss-protection?)))
(def app (app-handler
;; add your application routes here
[wiki-routes base-routes]
;; add custom middleware here
:middleware (load-middleware)
:ring-defaults (mk-defaults false)
;; add access rules here
:access-rules [{:redirect "/auth"
:rule user-access}]
;; serialize/deserialize the following data formats
;; available formats:
;; :json :json-kw :yaml :yaml-kw :edn :yaml-in-html
:formats [:json-kw :edn :transit-json]))
;; add your application routes here
[wiki-routes base-routes]
;; add custom middleware here
:middleware (load-middleware)
:ring-defaults (make-defaults true)
;; add access rules here
:access-rules [{:redirect "/auth"
:rule user-access}]
;; serialize/deserialize the following data formats
;; available formats:
;; :json :json-kw :yaml :yaml-kw :edn :yaml-in-html
:formats [:json-kw :edn :transit-json]))

View file

@ -41,11 +41,7 @@
[^String log-entry ^String file-path]
(timbre/info (format "searching '%s' for '%s'" log-entry file-path))
(cond
(not
(empty?
(filter
#(= (first %) file-path)
(:changed_files log-entry))))
(seq (filter (fn* [p1__341301#] (= (first p1__341301#) file-path)) (:changed_files log-entry)))
log-entry))
@ -88,7 +84,7 @@
(try
(.reset result reader (.getId tree))
(finally
(.release reader)
(.close reader)
(.dispose walk)))
result))
@ -121,7 +117,7 @@
new-parse)
(PathFilter/create file-path))
out))))
(.toString out))))
(str out))))
(defn fetch-version
@ -144,4 +140,4 @@
(throw (IllegalStateException.
(str "Did not find expected file '" file-path "'"))))
(.copyTo (.open repo (.getObjectId tw 0)) out)
(.toString out)))
(str out)))

View file

@ -2,12 +2,16 @@
(ns ^{:doc "Render a page as HTML."
:author "Simon Brooke"}
smeagol.layout
(:require [clojure.string :as s]
(:require [clojure.java.io :as cjio]
[clojure.string :as s]
[compojure.response :refer [Renderable]]
[environ.core :refer [env]]
[hiccup.core :refer [html]]
[ring.util.anti-forgery :refer [anti-forgery-field]]
[ring.util.response :refer [content-type response]]
[selmer.parser :as parser]
[smeagol.configuration :refer [config]]
[smeagol.sanity :refer :all]
[smeagol.util :as util]
[taoensso.timbre :as timbre]))
@ -51,23 +55,30 @@
(deftype RenderableTemplate [template params]
Renderable
(render [this request]
(content-type
(->> (assoc params
(keyword (s/replace template #".html" "-selected")) "active"
:i18n (util/get-messages request)
:dev (env :dev)
:servlet-context
(if-let [context (:servlet-context request)]
;; If we're not inside a serlvet environment (for
;; example when using mock requests), then
;; .getContextPath might not exist
(try (.getContextPath context)
(try
(content-type
(->> (assoc params
(keyword (s/replace template #".html" "-selected")) "active"
:i18n (util/get-messages request)
:dev (env :dev)
:servlet-context
(if-let [context (:servlet-context request)]
;; If we're not inside a serlvet environment (for
;; example when using mock requests), then
;; .getContextPath might not exist
(try (.getContextPath context)
(catch IllegalArgumentException _ context))))
(parser/render-file (str template-path template))
response)
"text/html; charset=utf-8")))
(parser/render-file (str template-path template))
response)
"text/html; charset=utf-8")
(catch Exception any
(show-sanity-check-error any)))))
(defn render [template & [params]]
(RenderableTemplate. template params))
(defn render
[template & [params]]
(try
(RenderableTemplate. template params)
(catch Exception any
(show-sanity-check-error any))))

View file

@ -40,13 +40,11 @@
(def development-middleware
[wrap-error-page
wrap-exceptions
wrap-anti-forgery])
wrap-exceptions])
(def production-middleware
[#(wrap-internal-error % :log (fn [e] (timbre/error e)))
wrap-anti-forgery])
[#(wrap-internal-error % :log (fn [e] (timbre/error e)))])
(defn load-middleware []

View file

@ -17,6 +17,7 @@
[smeagol.history :as hist]
[smeagol.layout :as layout]
[smeagol.routes.admin :as admin]
[smeagol.sanity :refer [show-sanity-check-error]]
[smeagol.util :as util]
[smeagol.uploads :as ul]
[taoensso.timbre :as timbre]))
@ -83,25 +84,29 @@
([request]
(edit-page request (util/get-message :default-page-title request) ".md" "edit.html" "_edit-side-bar.md"))
([request default suffix template side-bar]
(let [params (keywordize-keys (:params request))
src-text (:src params)
page (or (:page params) default)
file-name (str page suffix)
file-path (cjio/file util/content-dir file-name)
exists? (.exists (cjio/as-file file-path))
user (session/get :user)]
(if (not exists?)
(timbre/info (format "File '%s' not found; creating a new file" file-path))
(timbre/info (format "Opening '%s' for editing" file-path)))
(cond src-text (process-source params suffix request)
true
(layout/render template
(merge (util/standard-params request)
{:title (str (util/get-message :edit-title-prefix request) " " page)
:page page
:side-bar (md->html (slurp (cjio/file util/content-dir side-bar)))
:content (if exists? (slurp file-path) "")
:exists exists?}))))))
(or
(show-sanity-check-error)
(let [params (keywordize-keys (:params request))
src-text (:src params)
page (or (:page params) default)
file-name (str page suffix)
file-path (cjio/file util/content-dir file-name)
exists? (.exists (cjio/as-file file-path))
user (session/get :user)]
(if-not
exists?
(timbre/info
(format "File '%s' not found; creating a new file" file-path))
(timbre/info (format "Opening '%s' for editing" file-path)))
(cond src-text (process-source params suffix request)
true
(layout/render template
(merge (util/standard-params request)
{:title (str (util/get-message :edit-title-prefix request) " " page)
:page page
:side-bar (md->html (slurp (cjio/file util/content-dir side-bar)))
:content (if exists? (slurp file-path) "")
:exists exists?})))))))
(defn edit-css-page
@ -113,21 +118,23 @@
(defn wiki-page
"Render the markdown page specified in this `request`, if any. If none found, redirect to edit-page"
[request]
(let [params (keywordize-keys (:params request))
page (or (:page params) (util/get-message :default-page-title "Introduction" request))
file-name (str page ".md")
file-path (cjio/file util/content-dir file-name)
exists? (.exists (clojure.java.io/as-file file-path))]
(cond exists?
(do
(timbre/info (format "Showing page '%s' from file '%s'" page file-path))
(layout/render "wiki.html"
(merge (util/standard-params request)
{:title page
:page page
:content (md->html (slurp file-path))
:editable true})))
true (response/redirect (str "/edit?page=" page)))))
(or
(show-sanity-check-error)
(let [params (keywordize-keys (:params request))
page (or (:page params) (util/get-message :default-page-title "Introduction" request))
file-name (str page ".md")
file-path (cjio/file util/content-dir file-name)
exists? (.exists (clojure.java.io/as-file file-path))]
(cond exists?
(do
(timbre/info (format "Showing page '%s' from file '%s'" page file-path))
(layout/render "wiki.html"
(merge (util/standard-params request)
{:title page
:page page
:content (md->html (slurp file-path))
:editable true})))
true (response/redirect (str "/edit?page=" page))))))
(defn history-page
@ -179,7 +186,7 @@
(timbre/info (format "Showing version '%s' of page '%s'" version page))
(layout/render "wiki.html"
(merge (util/standard-params request)
{:title (str (util/get-message :vers-col-hdr request) " " version " of " page)
{:title (str (util/get-message :vers-col-hdr request) " " version " " (util/get-message :of request) " " page)
:page page
:content (md->html content)}))))
@ -194,35 +201,48 @@
(timbre/info (format "Showing diff between version '%s' of page '%s' and current" version page))
(layout/render "wiki.html"
(merge (util/standard-params request)
{:title (str (util/get-message :diff-title-prefix request)" " version " of " page)
{:title
(str
(util/get-message :diff-title-prefix request)
" "
version
" "
(util/get-message :of request)
" "
page)
:page page
:content (d2h/diff2html (hist/diff util/content-dir file-name version))}))))
:content (d2h/diff2html
(hist/diff util/content-dir file-name version))}))))
(defn auth-page
"Render the auth page"
[request]
(let [params (keywordize-keys (:form-params request))
username (:username params)
password (:password params)
action (:action params)
user (session/get :user)
redirect-to (or (:redirect-to params) "/wiki")]
(cond
(= action (util/get-message :logout-label request))
(do
(timbre/info (str "User " user " logging out"))
(session/remove! :user)
(response/redirect redirect-to))
(and username password (auth/authenticate username password))
(do
(session/put! :user username)
(response/redirect redirect-to))
true
(layout/render "auth.html"
(merge (util/standard-params request)
{:title (if user (str (util/get-message :logout-link request) " " user) (util/get-message :login-link request))
:redirect-to ((:headers request) "referer")})))))
(or
(show-sanity-check-error)
(let [params (keywordize-keys (:form-params request))
username (:username params)
password (:password params)
action (:action params)
user (session/get :user)
redirect-to (or (:redirect-to params) "/wiki")]
(cond
(= action (util/get-message :logout-label request))
(do
(timbre/info (str "User " user " logging out"))
(session/remove! :user)
(response/redirect redirect-to))
(and username password (auth/authenticate username password))
(do
(session/put! :user username)
(response/redirect redirect-to))
true
(layout/render "auth.html"
(merge (util/standard-params request)
{:title (if user
(str (util/get-message :logout-link request) " " user)
(util/get-message :login-link request))
:redirect-to ((:headers request) "referer")}))))))
(defn passwd-page

112
src/smeagol/sanity.clj Normal file
View file

@ -0,0 +1,112 @@
(ns ^{:doc "Functions related to sanity checks and error reporting in conditions where the environment may not be sane."
:author "Simon Brooke"}
smeagol.sanity
(:require [clojure.java.io :as cjio]
[hiccup.core :refer [html]]
[smeagol.configuration :refer [config]]
[smeagol.util :as util]
[taoensso.timbre :as timbre]))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;
;;;; Smeagol: a very simple Wiki engine.
;;;;
;;;; This program is free software; you can redistribute it and/or
;;;; modify it under the terms of the GNU General Public License
;;;; as published by the Free Software Foundation; either version 2
;;;; of the License, or (at your option) any later version.
;;;;
;;;; This program is distributed in the hope that it will be useful,
;;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;;; GNU General Public License for more details.
;;;;
;;;; You should have received a copy of the GNU General Public License
;;;; along with this program; if not, write to the Free Software
;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
;;;; USA.
;;;;
;;;; Copyright (C) 2014 Simon Brooke
;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn check-content-dir
"Check that the content directory exists and is populated. Throw exception
if not."
[]
(try
(let [directory (cjio/as-file util/content-dir)]
(if
(.isDirectory directory)
true
(throw (Exception. (str "Content directory '" util/content-dir "' is not a directory"))))
(if
(.canWrite directory)
true
(throw (Exception. (str "Content directory '" util/content-dir "' is not writable")))))
(catch Exception any
(throw (Exception. (str "Content directory '" util/content-dir "' does not exist") any))))
(try
(doall
(map
#(let
[path (cjio/file util/content-dir %)]
(timbre/info "Checking the existence of " path)
(slurp path))
["_side-bar.md" "_edit-side-bar.md" "_header.md"]))
(timbre/info "Content directory '" util/content-dir "' check completed.")
(catch Exception any
(throw (Exception. (str "Content directory '" util/content-dir "' is not initialised") any)))))
(defn- raw-sanity-check-installation
"Actually do the sanity check."
[]
(timbre/info "Running sanity check")
(check-content-dir)
(config :test)
(timbre/info "Sanity check completed"))
;;; We memoise the sanity check so that although it is called for every wiki
;;; page, it is only actually evaluated once.
(def sanity-check-installation (memoize raw-sanity-check-installation))
(defn- get-causes
"Get the causes of this `error`, if it is an Exception."
[error]
(if
(instance? Exception error)
(cons error (get-causes (.getCause error)))
'()))
(defn show-sanity-check-error
"Generate an error page in a way which should work even when everything else is broken.
If no argument is passed, run the sanity check and if it fails return page contents;
if `error` is passed, just return page content describing the error."
([error]
(html
[:html
[:head
[:title "Smeagol is not initialised correctly"]
[:link {:href "/content/stylesheet.css" :rel "stylesheet"}]]
[:body
[:header
[:h1 "Smeagol is not initialised correctly"]]
[:div {:id "error"}
[:p {:class "error"} (.getMessage error)]]
[:p "There was a problem launching Smeagol probably because of misconfiguration:"]
(apply
vector
(cons :ol
(map #(vector :li (.getMessage %))
(get-causes error))))
[:p "For more information please see documentation "
[:a {:href "https://github.com/journeyman-cc/smeagol/blob/develop/resources/public/content/Deploying%20Smeagol.md"} "here"]]]]))
([]
(try
(sanity-check-installation)
nil
(catch Exception any (show-sanity-check-error any)))))

View file

@ -51,18 +51,16 @@
:version (System/getProperty "smeagol.version")}))
(defn raw-get-messages
(defn- raw-get-messages
"Return the most acceptable messages collection we have given the
`Accept-Language` header in this `request`."
[request]
(merge
(i18n/get-messages
((:headers request) "accept-language")
;; (cjio/file (io/resource-path) "i18n")
"i18n"
"en-GB")
config)
)
config))
(def get-messages (memoize raw-get-messages))