All working
TODO: Could do with some tests, to guard for regressions.
This commit is contained in:
parent
7c8b1f2fea
commit
0dd5bd7477
79
README.md
79
README.md
|
@ -4,17 +4,37 @@ Simple command line utility to convert CSV files to EDN. Assumes the first row o
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
Download from http://example.com/FIXME.
|
[](https://clojars.org/csv2edn)
|
||||||
|
|
||||||
## Usage
|
### Leiningen/Boot
|
||||||
|
|
||||||
FIXME: explanation
|
[csv2edn "0.1.0"]
|
||||||
|
|
||||||
$ java -jar csv2edn-0.1.0-standalone.jar [args]
|
### Clojure CLI/deps.edn
|
||||||
|
|
||||||
## Options
|
csv2edn {:mvn/version "0.1.0"}
|
||||||
|
|
||||||
Where args are:
|
### Gradle
|
||||||
|
|
||||||
|
compile 'csv2edn:csv2edn:0.1.0'
|
||||||
|
|
||||||
|
### Maven
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>csv2edn</groupId>
|
||||||
|
<artifactId>csv2edn</artifactId>
|
||||||
|
<version>0.1.0</version>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
## Usage: as a standalone commandline tool
|
||||||
|
|
||||||
|
To run from the command line:
|
||||||
|
|
||||||
|
$ java -jar csv2edn-0.1.0-standalone.jar [options]
|
||||||
|
|
||||||
|
### Options
|
||||||
|
|
||||||
|
Where options are:
|
||||||
|
|
||||||
-h, --help Print this message and exit.
|
-h, --help Print this message and exit.
|
||||||
-i, --input INPUT The path name of the CSV file to be read.
|
-i, --input INPUT The path name of the CSV file to be read.
|
||||||
|
@ -24,10 +44,7 @@ Where args are:
|
||||||
-s, --separator SEPARATOR The character to treat as a separator in the
|
-s, --separator SEPARATOR The character to treat as a separator in the
|
||||||
CSV file. If not specified, comma will be used by default.
|
CSV file. If not specified, comma will be used by default.
|
||||||
|
|
||||||
|
### Examples
|
||||||
FIXME: listing of options this app accepts.
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
The simplest possible use is to simply use this in a pipeline:
|
The simplest possible use is to simply use this in a pipeline:
|
||||||
|
|
||||||
|
@ -45,13 +62,45 @@ or
|
||||||
$ java -jar csv2edn-0.1.0-standalone.jar \
|
$ java -jar csv2edn-0.1.0-standalone.jar \
|
||||||
--input path/to/file.csv --output path/to/file.edn
|
--input path/to/file.csv --output path/to/file.edn
|
||||||
|
|
||||||
|
## Usage: as a library
|
||||||
|
|
||||||
|
To use this package as a library in your own code:
|
||||||
|
|
||||||
|
(:require [csv2edn.csv2edn :refer [csv->edn *options*]])
|
||||||
|
|
||||||
|
And then pass either filenames or a reader and writer respectively to the
|
||||||
|
function `cvs->edn`, thus:
|
||||||
|
|
||||||
|
(cvs->edn *in* *out*)
|
||||||
|
;; reads standard input and writes to standard output
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
(cvs->edn "path/to/data.csv" "path/to/data.edn")
|
||||||
|
|
||||||
|
In either case, if successful, the function returns a possibly-lazy sequence
|
||||||
|
of maps representing the second and subsequent lines in the file (the first
|
||||||
|
line being assumed to contain column headers).
|
||||||
|
|
||||||
|
If you do not wish to write an EDN file, simply do not pass the second
|
||||||
|
argument.
|
||||||
|
|
||||||
|
If you want to use a non-standard separator charactor, e.g. bar rather than
|
||||||
|
comma, you should rebind `*options*`:
|
||||||
|
|
||||||
|
(binding [*options* (merge *options* {:separator \|})]
|
||||||
|
(cvs->edn "path/to/data.csv" "path/to/data.edn"))
|
||||||
|
|
||||||
|
Or else pass the separator character you want to use as a third argument:
|
||||||
|
|
||||||
|
(cvs->edn "path/to/data.csv" "path/to/data.edn" \|)
|
||||||
|
|
||||||
|
|
||||||
### Bugs
|
### Bugs
|
||||||
|
|
||||||
...
|
Not strictly a bug, but this does not yet work in ClojureScript, because it
|
||||||
|
depends on `clojure.data.csv`, which also doesn't. It would be good to fix
|
||||||
### Any Other Sections
|
this.
|
||||||
### That You Think
|
|
||||||
### Might be Useful
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|
|
@ -13,8 +13,6 @@
|
||||||
[lein-release "1.0.5"]]
|
[lein-release "1.0.5"]]
|
||||||
:profiles {:uberjar {:aot :all}}
|
:profiles {:uberjar {:aot :all}}
|
||||||
|
|
||||||
:lein-release {:deploy-via :clojars}
|
|
||||||
|
|
||||||
;; `lein release` doesn't play nice with `git flow release`. Run `lein release` in the
|
;; `lein release` doesn't play nice with `git flow release`. Run `lein release` in the
|
||||||
;; `develop` branch, then reset the `master` branch to the release tag.
|
;; `develop` branch, then reset the `master` branch to the release tag.
|
||||||
|
|
||||||
|
|
|
@ -1,11 +1,14 @@
|
||||||
(ns csv2edn.core
|
(ns ^{:doc "Conversion from CVS to EDN: standalone command line interface."
|
||||||
|
:author "Simon Brooke"}
|
||||||
|
csv2edn.core
|
||||||
(:require [clojure.java.io :as io]
|
(:require [clojure.java.io :as io]
|
||||||
[clojure.tools.cli :refer [parse-opts]]
|
[clojure.tools.cli :refer [parse-opts]]
|
||||||
[csv2edn.csv2edn :refer [csv2edn *options*]])
|
[csv2edn.csv2edn :refer [csv->edn *options*]])
|
||||||
(:gen-class))
|
(:gen-class))
|
||||||
|
|
||||||
|
|
||||||
(def cli-options
|
(def cli-options
|
||||||
|
"Command line argument specification."
|
||||||
[["-h" "--help" "Print this message and exit."]
|
[["-h" "--help" "Print this message and exit."]
|
||||||
["-i" "--input INPUT" "The path name of the CSV file to be read.
|
["-i" "--input INPUT" "The path name of the CSV file to be read.
|
||||||
If no input file is specified, input will be read from standard input."
|
If no input file is specified, input will be read from standard input."
|
||||||
|
@ -23,7 +26,7 @@
|
||||||
|
|
||||||
|
|
||||||
(defn -main
|
(defn -main
|
||||||
"I don't do a whole lot ... yet."
|
"Parse command line arguments if any, run the conversion, and exit."
|
||||||
[& opts]
|
[& opts]
|
||||||
(let [args (parse-opts opts cli-options)]
|
(let [args (parse-opts opts cli-options)]
|
||||||
(println
|
(println
|
||||||
|
@ -32,10 +35,10 @@
|
||||||
(:summary args) "")
|
(:summary args) "")
|
||||||
(if (:errors args)
|
(if (:errors args)
|
||||||
(apply str (interpose "; " (:errors args))) ""))
|
(apply str (interpose "; " (:errors args))) ""))
|
||||||
(if-not (:help (:options args))
|
(if-not (or (:errors args) (:help (:options args)))
|
||||||
(binding [*options* (:options args)]
|
(binding [*options* (:options args)]
|
||||||
(doall
|
(doall
|
||||||
(csv2edn
|
(csv->edn
|
||||||
(or
|
(or
|
||||||
(:input *options*)
|
(:input *options*)
|
||||||
(io/reader *in*))
|
(io/reader *in*))
|
||||||
|
|
|
@ -1,10 +1,14 @@
|
||||||
(ns csv2edn.csv2edn
|
(ns ^{:doc "Conversion from CVS to EDN."
|
||||||
|
:author "Simon Brooke"} csv2edn.csv2edn
|
||||||
(:require [clojure.data.csv :as csv]
|
(:require [clojure.data.csv :as csv]
|
||||||
[clojure.java.io :as io]
|
[clojure.java.io :as io]
|
||||||
[clojure.pprint :refer [pprint]]
|
[clojure.pprint :refer [pprint]]
|
||||||
[clojure.string :refer [lower-case]]))
|
[clojure.string :refer [lower-case]]))
|
||||||
|
|
||||||
(def ^:dynamic *options* {:separator \,})
|
(def ^:dynamic *options*
|
||||||
|
"Defaults for options used in conversion (essentially, `:separator` is `,`,
|
||||||
|
much as you'd expect."
|
||||||
|
{:separator \,})
|
||||||
|
|
||||||
(defn maybe-read
|
(defn maybe-read
|
||||||
"If a string represents an integer or real, we'd really like to have that
|
"If a string represents an integer or real, we'd really like to have that
|
||||||
|
@ -14,19 +18,29 @@
|
||||||
(read-string s)
|
(read-string s)
|
||||||
(catch Exception _ s)))
|
(catch Exception _ s)))
|
||||||
|
|
||||||
(defn csv2edn
|
(defn csv->edn
|
||||||
"Read a CSV stream from the reader or filename `in`, and write corresponding
|
"Read a CSV stream from the reader or filename `from`, and write corresponding
|
||||||
EDN to the reader or filname `out`. Assume column headers are in row 1"
|
EDN to the reader or filname `to`, using the separator character `sep`. Assume
|
||||||
[from to]
|
column headers are in row 1. If `from` is `nil` or not supplied, do not write.
|
||||||
|
It `sep` is not supplied, take the `:separator` value from `*options*`. Returns
|
||||||
|
a (possibly-lazy) sequence of maps."
|
||||||
|
([from] (csv->edn from nil))
|
||||||
|
([from to] (csv->edn from to (:separator *options*)))
|
||||||
|
([from to sep]
|
||||||
(let [data (with-open [reader (if (string? from)(io/reader from) from)]
|
(let [data (with-open [reader (if (string? from)(io/reader from) from)]
|
||||||
(doall
|
(doall
|
||||||
(csv/read-csv
|
(csv/read-csv
|
||||||
reader
|
reader
|
||||||
:separator (:separator *options*))))
|
:separator (first (str sep)))))
|
||||||
headers (map #(keyword (lower-case %)) (first data))]
|
headers (map #(keyword (lower-case %)) (first data))]
|
||||||
|
(let [result (map
|
||||||
|
#(zipmap headers (map maybe-read %))
|
||||||
|
(rest data))]
|
||||||
|
(if to
|
||||||
(with-open [writer (if (string? to) (io/writer to) to)]
|
(with-open [writer (if (string? to) (io/writer to) to)]
|
||||||
(pprint
|
(pprint
|
||||||
(map
|
result
|
||||||
#(zipmap headers (map maybe-read %))
|
writer)))
|
||||||
(rest data))
|
result))))
|
||||||
writer))))
|
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue