Social media features which users want which [Mastodon](https://en.wikipedia.org/wiki/Mastodon_(social_network)) does not provide, or provides poorly.
+I do not know what I am doing; I am learning, and playing. Nothing in this document should be treated as good advice; it simply relates to the current state of my knowledge.
This document is intended to provide an extension vocabulary for ActivityStreams documents, which provides vocabulary for categorising and describing faults in ActivityPub documents.
The motivation is to be able to serialise a validation report on an ActivityPub document as an ActivityStreams document.
diff --git a/docs/codox/dog-and-duck.quack.picky.constants.html b/docs/codox/dog-and-duck.quack.picky.constants.html new file mode 100644 index 0000000..330d1fc --- /dev/null +++ b/docs/codox/dog-and-duck.quack.picky.constants.html @@ -0,0 +1,12 @@ + +Constants supporting the picky validator.
The URI of the context of an ActivityStreams object is expected to be this literal string.
The set of types we will accept as actors.
+There’s an explicit set of allowed actor types.
The Clojure reader barfs on :@context, although it is in principle a valid keyword. So we’ll make it once, here, to make the code more performant and easier to read.
Severity of faults found, as follows:
+:info not actually a fault, but an issue noted during validation;:minor things which I consider to be faults, but which don’t actually breach the spec;:should instances where the spec says something SHOULD be done, which isn’t;:must instances where the spec says something MUST be done, which isn’t;:critical instances where I believe the fault means that the object cannot be meaningfully processed.The URI of the context of a validation fault report object shall be this literal string.
The set of types we will accept as verbs.
+There’s an explicit set of allowed verb types.
Control variables for the picky validator.
If true, references to objects in fields will be reified and validated. If false, they won’t, but an :info level fault report will be generated.
There are several things in the spec which, in a document, may correctly be either
+Obviously to fully validate a document we ought to reify all the refs and check that they are themselves valid, but
+a. in some of the published test documents the URIs do not reference a valid document; b. there will be performance costs to reifying all the refs; c. in perverse cases, reifying refs might result in runaway recursion.
+TODO: I think that in production this should default to true.
The severity at which the binary validator will return false.
In practice documents seen in the wild do not typically appear to be fully valid, and this does not matter. This allows the sensitivity of the binary validator (dog-and-duck.quack.quack) to be tuned. It’s in this (dog-and-duck.quack.picky) namespace, not that one, because this namespace is where concerns about severity are handled.
Narrative values for fault reports of specific types, used by the picky validator.
Fault-finder for ActivityPub documents.
+Fault-finder for ActivityPub documents.
Generally, each -faults function will return:
nil if no faults were found;:narrative whose value shall be a natural language description of the fault type.Note that the reason for the :fault property is to be able to have a well known place, linked to from the @context URL, which allows narratives for each fault type to be served in as many natural languages as possible.
The idea further is that it should ultimately be possible to serialise a fault report as a document which in its own right conforms to the ActivityStreams spec.
If true, references to objects in fields will be reified and validated. If false, they won’t, but an :info level fault report will be generated.
There are several things in the spec which, in a document, may correctly be either
-Obviously to fully validate a document we ought to reify all the refs and check that they are themselves valid, but
-a. in some of the published test documents the URIs do not reference a valid document; b. there will be performance costs to reifying all the refs; c. in perverse cases, reifying refs might result in runaway recursion.
-TODO: I think that in production this should default to true.
The severity at which the binary validator will return false.
In practice documents seen in the wild do not typically appear to be fully valid, and this does not matter. This allows the sensitivity of the binary validator (dog-and-duck.quack.quack) to be tuned. It’s in this (dog-and-duck.quack.picky) namespace, not that one, because this namespace is where concerns about severity are handled.
As base-activity-required-properties, except that the type of the object is restricted.
Properties activities should have, keyed by activity type. Values are maps of the format of base-activity-required-properties, q.v.
(activity-type-faults x)(activity-type-faults x type)Return a list of faults found in the activity x; if type is also specified, it should be a string naming a specific activity type for which checks should be performed.
Some specific activity types have specific requirements which are not requirements.
The URI of the context of an ActivityStreams object is expected to be this literal string.
(actor-faults x)Return a list of faults found in actor x, or nil if none are.
The set of types we will accept as actors.
-There’s an explicit set of allowed actor types.
Properties most activities should have. Values are validating functions, each.
-See https://www.w3.org/TR/activitystreams-vocabulary/#dfn-activity
(coll-object-reference-or-fault value expected-type severity token)As object-reference-or-fault, except value argument may also be a list of objects and/or object references.
The Clojure reader barfs on :@context, although it is in principle a valid keyword. So we’ll make it once, here, to make the code more performant and easier to read.
(context? x)Returns true iff x quacks like an ActivityStreams context, else false.
A context is either 1. the URI (actually an IRI) activitystreams-context-uri, or 2. a collection comprising that URI and a map.
(filter-severity reports severity)Return a list of reports taken from these reports where the severity of the report is greater than this or equal to this severity.
(has-activity-type? x)Return true if the object x has a type which is an activity type, else false.
(has-actor-type? x)Return true if the object x has a type which is an actor type, else false.
(has-context? x)True if x is an ActivityStreams object with a valid context, else false.
(has-type-or-fault x acceptable severity token)If object x has a :type value which is acceptable, return nil; else return a fault object with this severity and token.
acceptable may be passed as either nil, a string, or a set of strings. If acceptable is nil, no type specific tests will be performed.
(has-type? x type)Return true if object x has type type, else false.
The values of type fields of ActivityStreams objects may be lists; they are considered to have a type if the type token is a member of the list.
Properties intransitive activities should have.
-See https://www.w3.org/TR/activitystreams-vocabulary/#dfn-intransitiveactivity
(link-faults x)Return a list of faults found in the link x, or nil if none are found.
(make-fault-object severity fault)Return a fault object with these severity, fault and narrative values.
An ActivityPub object MUST have a globally unique ID. Whether this is meaningful depends on whether we persist fault report objects and serve them, which at present I have no plans to do.
(object-faults x)(object-faults x expected-type)Return a list of faults found in object x, or nil if none are.
If expected-type is also passed, verify that x has expected-type. expected-type may be passed as a string or as a set of strings.
(object-reference-or-faults value expected-type severity token)If this value is either
The idea further is that it should ultimately be possible to serialise a fault report as a document which in its own right conforms to the ActivityStreams spec.
As base-activity-required-properties, except that the type of the object is restricted.
Properties activities should have, keyed by activity type. Values are maps of the format of base-activity-required-properties, q.v.
(activity-type-faults x)(activity-type-faults x type)Return a list of faults found in the activity x; if type is also specified, it should be a string naming a specific activity type for which checks should be performed.
Some specific activity types have specific requirements which are not requirements.
(actor-faults x)Return a list of faults found in actor x, or nil if none are.
Properties most activities should have. Values are validating functions, each.
+See https://www.w3.org/TR/activitystreams-vocabulary/#dfn-activity
(coll-object-reference-or-fault value expected-type severity token)As object-reference-or-fault, except value argument may also be a list of objects and/or object references.
Properties intransitive activities should have.
+See https://www.w3.org/TR/activitystreams-vocabulary/#dfn-intransitiveactivity
(link-faults x)A link object is required to have an href property. It may have all of rel | mediaType | name | hreflang | height | width | preview but I think they’re all optional.
(object-faults x)(object-faults x expected-type)Return a list of faults found in object x, or nil if none are.
If expected-type is also passed, verify that x has expected-type. expected-type may be passed as a string or as a set of strings.
(object-reference-or-faults value expected-type severity token)If this value is either
expected-type;expected-type; orand no faults are returned from validating the linked object, then return nil; else return a sequence comprising a fault object with this severity and token, prepended to the faults returned.
As with has-type-or-fault (q.v.), expected-type may be passed as a string or as a set of strings.
NOTE THAT if *reify-refs* is false, referenced objects will not actually be checked.
(persistent-object-faults x)Return a list of faults found in persistent object x, or nil if none are.
Severity of faults found, as follows:
-:info not actually a fault, but an issue noted during validation;:minor things which I consider to be faults, but which don’t actually breach the spec;:should instances where the spec says something SHOULD be done, which isn’t;:must instances where the spec says something MUST be done, which isn’t;:critical instances where I believe the fault means that the object cannot be meaningfully processed.(string-or-fault value severity token)(string-or-fault value severity token pattern)If this value is not a string, return a fault object with this severity and token, else nil. If pattern is also passed, it is expected to be a Regex, and the fault object will be returned unless value matches the pattern.
(uri-or-fault u severity if-missing-token)(uri-or-fault u severity if-missing-token if-invalid-token)If u is not a valid URI, return a fault object with this severity and if-invalid-token. If it’s nil, return a fault object with this severity and if-missing-token. Otherwise return nil.
The URI of the context of a validation fault report object shall be this literal string.
(verb-type? x)true if x, a string, represents a recognised ActivityStreams activity type.
The set of types we will accept as verbs.
-There’s an explicit set of allowed verb types.
NOTE THAT if *reify-refs* is false, referenced objects will not actually be checked.
(persistent-object-faults x)(persistent-object-faults x types severity token)Return a list of faults found in persistent object x, or nil if none are.
(string-or-fault value severity token)(string-or-fault value severity token pattern)If this value is not a string, return a fault object with this severity and token, else nil. If pattern is also passed, it is expected to be a Regex, and the fault object will be returned unless value matches the pattern.
(uri-or-fault u severity if-missing-token)(uri-or-fault u severity if-missing-token if-invalid-token)If u is not a valid URI, return a fault object with this severity and if-invalid-token. If it’s nil, return a fault object with this severity and if-missing-token. Otherwise return nil.
TODO: write docs
Utility functions supporting the picky validator
(concat-non-empty & lists)Quick function to replace the pattern (nil-if-empty (remove nil? (concat …))) which I’m using a lot!
(context? x)Returns true iff x quacks like an ActivityStreams context, else false.
A context is either 1. the URI (actually an IRI) activitystreams-context-uri, or 2. a collection comprising that URI and a map.
(filter-severity reports severity)Return a list of reports taken from these reports where the severity of the report is greater than this or equal to this severity.
(has-activity-type? x)Return true if the object x has a type which is an activity type, else false.
(has-actor-type? x)Return true if the object x has a type which is an actor type, else false.
(has-context? x)True if x is an ActivityStreams object with a valid context, else false.
(has-type-or-fault x acceptable severity token)If object x has a :type value which is acceptable, return nil; else return a fault object with this severity and token.
acceptable may be passed as either nil, a string, or a set of strings. If acceptable is nil, no type specific tests will be performed.
(has-type? x type)Return true if object x has type type, else false.
The values of type fields of ActivityStreams objects may be lists; they are considered to have a type if the type token is a member of the list.
(make-fault-object severity fault)Return a fault object with these severity, fault and narrative values.
An ActivityPub object MUST have a globally unique ID. Whether this is meaningful depends on whether we persist fault report objects and serve them, which at present I have no plans to do.
(verb-type? x)true if x, a string, represents a recognised ActivityStreams activity type.
Validator for ActivityPub objects: if it walks like a duck, and it quacks like a duck…
+Validator for ActivityPub objects: if it walks like a duck, and it quacks like a duck…
**NOTE THAT the ActivityPub spec says
-Servers SHOULD validate the content they receive to avoid content spoofing attacks
but in practice ActivityPub content collected in the wild bears only a hazy relationship to the spec, so this is difficult. I suspect that I may have to implement a *strict* dynamic variable, so that users can toggle some checks off.
(activity? x)(activity? x severity)true iff x quacks like an activity, else false.
(actor-or-uri? x)true if x is either a URI or an actor.
TODO: I need to decide about whether to reify referenced objects before validation or after. After reification, every reference to an actor must be to an actor object, but before, may only be to a URI pointing to one.
(collection-page? x)true iff x quacks like a page in a paged collection, else false.
(collection? x object-type)(collection? x)true iff x quacks like a collection of type object-type, else false.
With one argument, will recognise plain collections and ordered collections, but (currently) not collection pages.
(link-or-uri? x)true iff x is either a URI or a link, else false.
There are several points in the specification where e.g. the :image property (if present) may be either a link or a URI.
(object? x)(object? x severity)Returns true iff x is recognisably an ActivityStreams object.
but in practice ActivityPub content collected in the wild bears only a hazy relationship to the spec, so this is difficult. I suspect that I may have to implement a *strict* dynamic variable, so that users can toggle some checks off.
(activity? x)(activity? x severity)true iff x quacks like an activity, else false.
(actor-or-uri? x)true if x is either a URI or an actor.
TODO: I need to decide about whether to reify referenced objects before validation or after. After reification, every reference to an actor must be to an actor object, but before, may only be to a URI pointing to one.
(collection-page? x)true iff x quacks like a page in a paged collection, else false.
(collection? x object-type)(collection? x)true iff x quacks like a collection of type object-type, else false.
With one argument, will recognise plain collections and ordered collections, but (currently) not collection pages.
(link-or-uri? x)true iff x is either a URI or a link, else false.
There are several points in the specification where e.g. the :image property (if present) may be either a link or a URI.
(object? x)(object? x severity)Returns true iff x is recognisably an ActivityStreams object.
NOTE THAT The ActivityStreams spec says:
All properties are optional (including the id and type)
@@ -18,5 +18,5 @@-Implementers SHOULD include the ActivityPub context in their object definitions
but in samples found in the wild they typically don’t.
(ordered-collection-page? x)true iff x quacks like a page in an ordered paged collection, else false.
(ordered-collection? x)true iff x quacks like an ordered collection, else false.
(persistent-object? x)(persistent-object? x severity)true iff x is a persistent object.
Transient objects in ActivityPub are not required to have an id key, but persistent ones must have a key, and it must be an IRI (but normally a URI).
(unordered-collection? x)true iff x quacks like an unordered collection, else false.
but in samples found in the wild they typically don’t.
(ordered-collection-page? x)true iff x quacks like a page in an ordered paged collection, else false.
(ordered-collection? x)true iff x quacks like an ordered collection, else false.
(persistent-object? x)(persistent-object? x severity)true iff x is a persistent object.
Transient objects in ActivityPub are not required to have an id key, but persistent ones must have a key, and it must be an IRI (but normally a URI).
(unordered-collection? x)true iff x quacks like an unordered collection, else false.
TODO: write docs
(clean json)Take this json input, and return a sequence of ActivityPub objects represented by it.
TODO: write docs
(clean json)Take this json input, and return a sequence of ActivityPub objects represented by it.
Scratchpad where I try to understand how to do this stuff.
Scratchpad where I try to understand how to do this stuff.
TODO: write docs
return the hostname of the current host.
+TODO: write docs
return the hostname of the current host.
Java’s methods for getting the hostname are quite startlingly slow, we do not want todo this repeatedly!
Get the process id of the current process.
OK, this is hacky as fuck, but I hope it works. The problem is that the way to get the process id has changed several times during the history of Java development, and the code for one version of Java won’t even compile in a different version.
A playground for hacking ActivityPub stuff.
To install, add the following dependency to your project or build file:
[dog-and-duck "0.1.0-SNAPSHOT"]
copied from Jahfer’s clj-activitypub library. If and when Jahfer issues a release of that library, this directory will be deleted and a dependency on that library will be added to the project.
Public variables and functions:
copied from Jahfer’s clj-activitypub library. If and when Jahfer issues a release of that library, this directory will be deleted and a dependency on that library will be added to the project.
Public variables and functions:
copied from Jahfer’s clj-activitypub library. If and when Jahfer issues a release of that library, this directory will be deleted and a dependency on that library will be added to the project.
Public variables and functions:
copied from Jahfer’s clj-activitypub library. If and when Jahfer issues a release of that library, this directory will be deleted and a dependency on that library will be added to the project.
Public variables and functions:
copied from Jahfer’s clj-activitypub library. If and when Jahfer issues a release of that library, this directory will be deleted and a dependency on that library will be added to the project.
Public variables and functions:
Fault-finder for ActivityPub documents.
Public variables and functions:
Validator for ActivityPub objects: if it walks like a duck, and it quacks like a duck…
Public variables and functions:
Scratchpad where I try to understand how to do this stuff.
Public variables and functions:
A playground for hacking ActivityPub stuff.
To install, add the following dependency to your project or build file:
[dog-and-duck "0.1.0-SNAPSHOT"]
Fault-finder for ActivityPub documents.
Public variables and functions:
Constants supporting the picky validator.
Public variables and functions:
Control variables for the picky validator.
Public variables and functions:
Narrative values for fault reports of specific types, used by the picky validator.
Public variables and functions:
Utility functions supporting the picky validator
Public variables and functions:
Validator for ActivityPub objects: if it walks like a duck, and it quacks like a duck…
Public variables and functions:
Scratchpad where I try to understand how to do this stuff.
Public variables and functions:
A Clojure library designed to implement the ActivityPub protocol, obviously.
NOTE THAT what Mastodon delivers to clients is not actually in ActivityStreams format; this seems to be an ad-hoc hack that’s just never been fixed and has therefore become a de-facto standard for communication between ActivityPub hosts and their clients.
+The ActivityPub spec starts by saying:
+++ActivityPub provides two layers:
++
+- A server to server federation protocol (so decentralized websites can share information)
+- A client to server protocol (so users, including real-world users, bots, and other automated processes, can communicate with ActivityPub using their accounts on servers, from a phone or desktop or web application or whatever).
+
I’m interested in driving much more functionality down to the the client, for example feed ordering and presentation. This would allow users, for example, to choose their own (or roll their own) feed-ordering algorithms.
My proposal would be to deliver exactly the same ActivityStreams format to my client as to other servers. There may be a valid reason for not doing this, but if there is I will discover it in due course.
+The ActivityStreams spec seems predicated on ‘always up’ communication between at least servers, which is perhaps why there is a two tier network of ‘servers’ and ‘clients’. It also depends on HTTPS certificates to identify servers, which implies it’s vulnerable to disruption by a hostile actor with the ability to revoke certificates.
+My own history with social media dates back to Usenet over UUCP, a system designed explicitly for intermittent low bandwidth connections; such a system is immensely resilient in the face of disruption to infrastructure.
+Social media is useful to concerted popular action in periods of disruption, whether in the case of civil ememrgency such as earthquakes, wild fires and floods, in the case of wars, or in the case of intrusive surveillance by authoritarian governments. But to be useful in such situations it needs to be resilient, and one of the things it needs to be resilient to is parts of the network being intermittently available, or requiring rerouting.
+In this I’m influenced by and hope to try to implement ideas from Ian Clarke’s Freenet and Tahrir projects, especially webs of trust.
+To be clear, it is important for The Old Dog and Duck to be able to interact with other existing ‘vanilla’ ActivityStreams implementations, but I hope to experiment with enhanced communication between Dog and Duck servers to provide more FreeNet-like resiliency.
NOTE THAT at the present stage all the proposed libraries are in one package, namely this package, but that it is proposed that in future they will form separate libraries in separate packages.