Putting Org mode on the Indieweb.
This manual corresponds to indie-org version 0.6.0.
Many of us maintain personal websites using Org Mode. An Org-generated static site has advantages over full-blown Content Management Systems: authoring pages in Org, working in Emacs Lisp, unparalleled control & customizability over the process, and the simplicity of not having to leave your Emacs environment to name a few.
However, that simplicity comes with costs such as fewer features; such a site will not be able to handle commenting or support user login, for instance. Furthermore, in an age of social media, an independent site can come to feel isolated altogether, existing outside the conversations taking place on sites like Twitter & Mastodon.
Enter the Indieweb: “The IndieWeb is a community of independent & personal websites connected by simple standards, based on the principles of: owning your domain & using it as your primary identity, publishing on your own site (optionally syndicating elsewhere), and owning your data.”
The Indieweb (See The Indieweb) offers protocols, idioms & services for connecting your site to those of others, and even to other ecosystems such as Twitter & Mastodon (“silos” in the Indieweb’s parlance).
indie-org
is an Emacs package for integrating an Org-generated,
static site with the Indieweb. This manual describes indie-org
for the interested site author. Throughout, it will reference a simple
static site, indie-org.sh. It is live on
the web, and its source is available at
Github. The reader may
wish to refer to the source while reading this manual.
Per indieweb.org: “the Indieweb is a collection of protocols for connecting to other independent sites, pushing your content to social media sites, collecting likes, comments & responses from other sites _back_ to yours, and many other things as well.”
Many of the Indieweb principles would likely appeal to the Emacs Org Mode user, and the Org Mode static site owner in particular:
The Org Mode static site owner has already taken a large step in this direction.
Nothing more need be said.
This manual is a step in that direction on the author’s part.
This section covers a few aspects of the Indieweb that are pertinent
to indie-org
.
Microformats are a fundamental building block of the Indieweb. “Designed for humans first and machines second, microformats are a set of simple, open data formats built upon existing and widely adopted standards”. They are a set of tags which one may add to one’s HTML to indicate semantic rather than formatting information. For instance, consider mentioning a person in your prose. In plain text, one could say:
Barnaby Walters
and in HTML, to indicate a location on the internet, one could say:
<a href="http://waterpigs.co.uk">Barnaby Walters</a>
Microformats2 adds additional information to that link, the information that this names a person, by adding a particular class to the HTML element:
<a class="h-card" href="http://waterpigs.co.uk">Barnaby Walters</a>
Microformats allow you to build structure, for instance giving this reference to a person both a location on the web and a photo:
<div class="h-card"> <p><img class="u-photo" href="/me.png" alt="" /></p> <p class="p-name"><a href="u-url" href="http://waterpigs.co.uk">Barnaby Walters</a></p> </div>
You can find a complete vocabulary here, and an on-line Microformats parser here.
Of particular note is that, as static site authors, the onus is on us to mark-up our HTML with microformts. We’ll use the page template in the Org Export back-end to do so (see below).
The Microformats Wiki is here and mf2py is a commonly used Python library for parsing Microformats.
In a world in which everyone owns their own identity, how shall we authenticate one another? OAuth 2.0 has become increasingly common, but suffers from a few shortcomings that make it unsuitable for use on the Indieweb.
In the first place, the protocol requires an app to register with the authorization server to be used (see the RFC, section 2). This makes sense when, say, building a Facebook app, or a Twitter app, but breaks-down in a world in which everyone is their own authorizing authority.
The second issue is simply that OAuth 2.0 is not an authentication protocol– it is a protocol designed to authorize applications to access resources on a principal’s behalf. Without augmenting the protocol in some way, the app will have no information that can identify the user on their app– just a grant of permissions to carry out certain operations on the resource provider.
This led Aaron Parecki and others to develop a related protocol– IndieAuth. Here, your identity is your domain name. When you wish to authenticate with a site, you no longer pick one of a few silos (“sign-in with Google”, “sign-in with Facebook”, and so forth), but instead provide your domain name. The site will fetch your page (or, at least, its head) to discover your IndieAuth provider endpoints.
From there, the flow largely proceeds in the same manner as OAuth with the additional bit that the authorization endpoint will provide not only a token, but a canonical URL identifying the user. The full authorization flow is described here.
This is an important step forward, but making everyone on the Indieweb responsible for implementating an authentication server would not be very practical. Fortunately, there are multiple standalone implementations available, beginning with IndieAuth.com. This is an IndieAuth authentication implementation that will allow one to authenticate using either one’s existing social media accounts or PGP keypair, even (or, perhaps, especially) when using a static site.
In order to integrate with IndieAuth.com, the site author needs to add
a link to each account to the home page with the attribute
rel="me"
, and ensure that those accounts list your home page in
their profiles:
<ul> <li> <a href="https://twitter.com/jdoe" rel="me">Twitter</a> </li> <li> <a href="https://github.com/jdoe" rel="me">Github</a> </li> ... </ul>
This manual will cover the details for a sample site See below.
Webmention is an open standard for distributed interactions among independent web sites.
Suppose Alice posts to her site. The content is presumably HTML, but could be something else (plain/text, say). Bob now wants to mention that content on his site. He makes a post (the source) that contains the URL of Alice’s post (the target). Again, the source is presumably an HTML document, but may be anything that can express an URL.
Bob, or more likely his publishing software, must now discover Alice’s
Webmention endpoint. He does this by fetching Alice’s document via
HTTP– if it has a Link
header with a rel value of
“webmention” that’s Alice’s webmention endpoint (this is how sending
webmentions of non-HTML documents is supported). If there is no such
header, and the document is HTML, Bob must look through the document
itself for <link>
or <a>
elements with rel values of
“webmention”– the first one in the document is Alice’s Webmention
endpoint.
Bob then posts x-www-form-urlencoded
source & target parameters
to Alice’s Webmention endpoint (the target parameter being Alice’s
post and the source being Bob’s mention thereof). Alice’s Webmention
endpoint will typically do some initial validation of the request, and
on success return either “201 Created” or “202 Accepted” to
indicate that further processing will take place asynchronously (in
the former case, the Location
header in the response will
include an URL at which the request processing status may be checked).
Asynchronously, Alice verifies the Webmention by issuing an HTTP
GET
on the source and verify that the source document contains
an exact match for the URL given in the target parameter of the
Webmention.
At this point, Alice may choose to publish content from Bob’s mention along with any other data it obtained (e.g. “Bob liked...”).
You can see an example of this in action (in conjunction with see POSSE) here.
“POSSE” stands for Publish on your Own Site, Syndicate Elsewhere” (or Everywhere). POSSE is less a formal protocol (like see Webmentions) than a feature you may want to build into your site.
The idea is that, after posting to one’s site, one then pushes copies of, or links to, that new post to assorted third-party sites (Twitter, for instance). This allows readers who prefer those sites to remain aware of your activity without leaving their prefered experience. This feature can be extended to collect responses to the POSSE’d content within the third-party sites (likes, replies and so forth) and feed that back to your site: a list of Tweets and/or Toots that have replied to you underneath your original post, for instance.
You can see an example of this in action (in conjunction with see Webmentions) here.
It stems from the Indieweb principle of owning your data rather than trusting it to a third-party site that may go away or simply decide to deny you access to your own content. It also stems from the hard fact that many of our friends & colleagues are still on such third-party sites such as Twitter or Mastodon: this is a way to connect your content thereto.
Implementations are generally bespoke; indie-org
supports this
for the site author via brid.gy (see
see Supporting POSSE).
This manual assumes that the reader is generally familiar with Org mode. If they are not, the Org Manual, site and Worg are excellent resources for getting started.
This section will instead concern itself with deeper explanations of a
few aspects of Org mode of particular interest to indie-org
:
the Org export process and the question of standardizing Org syntax
and its implications for static site generation based on Org mode
documents.
This subsection outlines the generic Org Export process with an eye
toward customizing it for our purposes. While we defer the
fully-customized implementation to See Customizing the Publication Process, it is here that we introduce indie-org.sh, a live example of an Org-generated static site built on
top of indie-org
.
The Org Export publication process, in broad terms, looks like this:
The main entrypoint to the Org Export facility is See org-publish-all.
org-publish-all
refers to the variable See org-publish-project-alist, which
describes our publication “project” to Org Mode. For each project
(the root project can have sub-projects), Org Export will iterate over
each file, copying it to its configured destination and, in general,
transforming the contents in some way.
Of particular note is the info
, or plist
argument which
is initialized at the beginning of the process of publishing each
project (in org-publish-file
) and is passed down through the
entire call stack. It begins as the project plist (i.e. the project
entry in org-publish-project-alist
) and is augmented at each
step. It is a property list that the different pieces of the
publication pipeline use to communicate, and indie-org
will use
it as well.
The process of transforming the Org Mode document to another format is
governed by the :publishing-function
attribute for each
project. It is invoked with the input filename, the publication
directory, and the project plist.
indie-org.sh
has two sub-projects salient to this discussion:
“pages” and “posts”. The former contains a few top-level pages
(home, about & so forth) whereas the latter contains all the site
posts. Each employs a lambda as its publishing-function
; the
lambdas simply provide one more argument to the true publishing
implementation, iosh/publish-page
. The additional argument is
the Org Export back-end to use ( see Back-end in Adding Export Back-ends ).
iosh/publish-page
passes that back-end on to
org-publish-org-to
and thence to org-export-to-file
and
org-export-as
, which is where the real work begins. The
function has a lot of functionality packed into its 171 lines; of note
to us, it will:
We introduce an Emacs Lisp file, indie-org.sh.el
to the project
where we can setup this boilerplate & invoke org-publish-all
:
(defun iosh/publish (prod) "Publish indie-org.sh to production if PROD is non-nil, locally else." ;; Define a derived backend to insert our template: (org-export-define-derived-backend 'iosh/page-html 'html :translate-alist '((template . iosh/page-html))) (org-export-define-derived-backend 'iosh/post-html 'html :translate-alist '((template . iosh/post-html))) ;; ... (let* ((publishing-root (concat (file-name-as-directory project-dir) "www")) (org-publish-project-alist `(("indie-org.sh" :components ("pages")) ("pages" :base-directory ,(concat project-dir "pages") :publishing-directory ,(concat project-dir "www") :publishing-function ,(list (lambda (plist filename pub-dir) (iosh/publish-page plist filename pub-dir 'iosh/page-html))) :html-doctype "html4-strict" ... ) ("posts" :base-directory ,(concat project-dir "posts") :publishing-directory ,(concat project-dir "www/posts") :publishing-function ,(list (lambda (plist filename pub-dir) (iosh/publish-page plist filename pub-dir 'iosh/post-html))) :html-doctype "html4-strict" ... )))) (org-publish-all t) ;; ... ))
It has been suggested to the author that he has built far too much functionality in Emacs Lisp, and that if one is interested in putting a staticly generated site on the Indieweb, it is better implemented as a purpose-built program written in a general-purpose programming language.
Perhaps. However, if one is committed to writing in Org Mode, that avenue of attack on the problem leads us to an interesting question: how can we parse Org Mode files outside of Emacs?
Today, the reference implementation of an Org Mode parser is... Emacs
Org Mode. The syntax is documented
here and the
implementation, in org-element.el
is unusually readable, but it
all exists within Emacs.
There are (numerous) parsers available, in a wide variety of languages, with various levels of fidelity to Org Mode claimed. There is an effort ongoing at the time of this writing on the emacs-orgmode list to standardize a set of test cases to make compliance more rigorous. Karl Voit independently gave a talk at EmacsConf 2021 proposing a specification which he called “Orgdown”.
So it may be that in the not too distant future, we will have Org Mode
parsers in general-purpose languages which can be validated against
some sort of specification, or at least a well-known test
suite. However, and this has been discussed at length on the
emacs-orgmode list, such a parser will either need to re-implement
much of Emacs or be incomplete: how should such a parser handle Babel,
for instance? Table formulas that invoke arbitrary Emacs Lisp
functions? The author uses a custom Org Mode property,
#+AUTODATE
in his documents; if set to t
, the
#+DATE
property will be updated every time the buffer is
saved– how should that be handled?
Some argue that such a specification could only cover the “static” portions of an Org Mode document, but it’s not clear to the author exactly how that would work in the case of the “dynamic” portions generating document content.
The answer is not clear to me at this time. However, having implemented the functionaly to support putting my own sites on the Indieweb, it seemed reasonable to make that functionality available to others.
We’ve seen that Org Mode is a flexible and extensible way to publish HTML from within Emacs. There are numerous ways in which one could push the resulting HTML to a server and thereby serve a static website. That said, a static site can be... limiting.
It is a way of broadcasting content to a passive audience, but no more. It provides no way for readers to respond to the content. Commenting as a service is available (from Disqus, for example), but in the first place such services always feel grafted-on to the host site, and in the second, if you’re going to the trouble to host your own site, are you really comfortable offloading the resulting conversations to a third party?
Additionally, although consumers are becoming increasingly chary of entrusting their online identities (and data) to sites like Twitter, the reality is that a great deal of public coversation continues to take place there (and on Facebook, Reddit, &c). A self-hosted, static web site offers no connection to such fora.
The indie-org.sh project was introduced in See The Export Process. This chapter will flesh-out the implementation details, including authoring Microformats. The next chapter, See Adding Indieweb Support, will cover truly adding the site to the Indieweb by adding things like Indieauth, Webmentions & POSSE support.
The site itself is structured with a single top-level page and a
series of posts beneath that (in the posts
directory). It is
hosted in an AWS S3 bucket fronted by Cloud Front.
The source for each page or post is an Org document. The top-level
page sources can be found in pages
and that for posts in
posts
. The Org documents are published to HTML via HTML export into a directory named
www
.
The publication process is run by invoking Emacs, telling it to load
the file lisp/indie-org.sh.el
, and telling it to invoke the
function iosh/publish
therein:
emacs --batch -l lisp/indie-org.sh.el --eval '(iosh/publish)'
When publishing to the live site, the www
directory is simply
copied to the S3 bucket using the aws
CLI.
Since it is tedious to type these commands over & over, a Gnu
Makefile
is provided: building the site locally is
make
and publishing it is make prod
.
The remainder of this chapter is a guided tour of the code intended to leave the reader in a position to put their own Org-generated static site on the Indieweb. Much of the work has been encapsulated in the indie-org Emacs package, but there is still a fair amount of per-site customization required.
We customize See The Export Process in order to produce microformats in our generated HTML and to send Webmentions & POSSE requests:
This section will broadly sketch the customizations; subsequent sections will detail implementing each aspect.
In order to emit See Microformats, we’re going to need to get our
fingers into the process of transcoding Org Mode to HTML; this entails
defining our own backend(s) and setting the
:publishing-function
for each project (see see Where & How to Author Microformats).
In order to support IndieAuth, we’ll need to generate a PGP keypair and/or setup accounts with one or more social media sites (see see Supporting IndieAuth).
In order to note webmentions made on each post, we’ll need to define new link types, so that we can indicate that we’d like to mention another post by authoring Org Mode such as:
as you can see [[mention:https://cool-site.org/cool-post.html][here]]...
In the export function for such link types, we’ll note the webmentions
in the page’s informational property list; then, in the project
:final-output
filter we can extract all of that page’s mentions
and add them to the site’s list (see see Supporting Webmentions.
POSSE is handled in the page template– since at this point the Org Mode document has been parsed, we can simply access them as properties and add them to the site’s list of POSSE requests.
In the foregoing, we’ve mentioned site-wide lists of webmentions made & POSSE requests. These lists reside in indie-org-state-v2. See See Publication Environments & State for details.
When one “publishes” an Org Export project, each file in the project
is copied to the destination directory and perhaps transformed in the
process (see Publishing in Org
Mode, OrgMode) The publication operation is determined by the
project’s :publishing-function
property, which defaults to
org-html-publish-to-html
. This function, in turn, invokes
org-publish-org-to
with the html
back-end.
The details of the Org Export process are determined by the selected
back-end (see Exporting in Org Mode, Org
Mode). Of particular interest is the template
member of each
back-end’s translate-alist
attribute– this shall be a lisp
form that receives the abstract syntax tree for each Org Mode document
published and is responsible for returning the text to be written to
the output file (HTML, in our case)– this is where we emit our
microformats.
This leaves us with a quandry: how shall we support different post types? One option would be to implement it in the template function: have the template extract the post-type document property, figure-out what sort of post we’re publishing, and emit the corresponding HTML.
Alternatively, we could implement it in the publication function– that is, have a different back-end for each post type. This is less convenient because you need to know the post type before the file is parsed (that is, all you’ve got at publication time is a filename).
So, let us define two Org Export projects, one for top-level pages on our site (home, about & so forth) and one for posts. The latter shall eventually support different post types.
At this point, we have a web site which, while static, is already
marked-up with Microformats. We can publish locally by saying
make
and to the web by saying make prod
. In the
next chapter, we’ll add assorted IndieWeb features to it.
Throughout indie-org
, we need a way to "name" pages on the
site. For the most part, indie-org
requires a page-key to
simply be a string that uniquely identifies each page on the site.
There are a few instances, however, in which we need a bit more
structure:
It would be possible to make this parameterizable by, say, providing
customizable variables holding functions for performing these two
operations. As a first implementation, however, indie-org
simply defines "page key" in the obivous way: as the page’s path in a
URL naming it on the site (i.e. if the page lives at
“https://foo.net/a/b/c.html” on your site, its page-key shall be
“a/b/c.html”).
Supporting various Indieweb features such as Webmentions & POSSE requires maintaining state regarding site publication. For instance, every time one publishes the site (whether to a local test server or to the live site) one now needs to scribble down new Webmentions that need to be sent for subsequent use. Incoming webmentions also need to be recorded upon receipt for potential use in subsequent publications.
indie-org
implicitly defines the idea of “publication
environment” to represent the various places to which one may wish to
publish. This is implicit because they are named by arbitrary symbols
chosen by the site author. For instance, one may choose to examine one’s
site after simply publishing HTML to the local filesystem in a browswer
with a “file:” URL, and name this publication environment :local
.
Perhaps one has an Apache instance running locally for testing
purposes: that could be named :staging
. The only assumption
made by indie-org
is that the default site is named
:prod
.
indie-org
associates each production environment with an
instance of the struct
indie-org-state-v2
.
The struct contains the fields:
While it is up to the site author to incorporate state into their
publication process, indie-org
provides basic utilities for
doing so.
Use
indie-org-make-publication-state
to create a new
indie-org-state-v2
instance,
indie-org-state-write
to write a property list mapping
publication environment to state to file, and
indie-org-state-read
to read it back out.
A site’s publication function might then look something like this:
(defun my/publish (prod) "Publish my site to production if PROD is non-nil, locally else." (indie-org-enable) (let* ((env (if prod :prod :local)) (pub-states (if (file-exists-p publication-state-file) (indie-org-state-read publication-state-file) `(:prod ,(indie-org-state-make) :local ,(indie-org-state-make)))) (publication-state (plist-get pub-states env))) ;; Setup the call to `org-publish-all'. (let* (...) (org-publish-all t) ;; Update state ... (plist-put pub-states env publication-state) ;; and write it all out to disk. (indie-org-state-write pub-states ".state"))))
Adding See IndieAuth support to one’s site requires no coding. This section will instead sketch the steps necessary to support IndieAuth to the extent necessary for adding additional IndieWeb features.
You’ll need to generate a PGP keypair to identify your site, if you
don’t already have a keypair you’d like to use. To generate a new
keypair, say gpg --gen-key
and follow the prompts. Think
ahead of time about the e-mail you’d like to associate with the new
keypair.
Once you’ve got your keypair, you can get its identifier by saying
gpg --list-keys
(the identifier is the long hexadecimal
string listed for each public key on your keyring). Say gpg
--output PATH-TO-YOUR-SOURCE/site-public.pgp --export KEYID
to put a
copy of your site’s public key somewhere in your site’s source project
directory.
Add a link to your home page like:
<link rel="pgpkey" href="/site-public.pgp>
To test, go to https://indieauth.com and scroll down to the “Try It!” section.
Note that IndieAuth.com supports neither Twitter nor Mastodon for authentication purposes.
You should see a green button labelled “GPG”
along with your domain & the name of your public key. Click that and
you’ll be presented with a string to sign with your private key
(demonstrating ownership of the private key). Copy the challenge text
and say echo 'CHALLENGE-TEXT'|gpg -u KEYID --clearsign --armor
. This
should produce output something like:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 eyJ0eXAiOiJKV1Qi... -----BEGIN PGP SIGNATURE----- iQGzBAEBCAAdFiEEr2ZwJM99NLbLOVkMca2Hl/Lwd1cFAmQqHdQACgkQca2Hl/Lw d1e1jwv/bpIrpr7+WNfD1xfiNzkq+PzbeeMT07B8kHo3ZKXJINB420jO3P+QqM8G S1WQF2XyhnzxKmo/ySk54HOV5iWZ62uBHIrn/Nn6YUBvVQUB6CiF0zeCvKrbreW/ 3omdbdLfCryPAMd120sQi8mQ5fDr798jq8Oq7QyIA4WusIh3ZesoDYboE4VJKryK ... -----END PGP SIGNATURE-----
Paste the entire messages into the box.
While IndieAuth does not support login via other sites, such as Twitter or Mastodon, IndieLogin does, and the services we’re going to be using use the latter, not the former.
In order to do this, the site author must mark-up their home page with
links to their social media profiles with the rel="me"
attribute, e.g.:
<link href="https://twitter.com/jdoe" rel="me">
On each service listed, ensure there’s a link back to the site’s home page.
Supported services:
This section will lay-out the process of adding See Webmentions support to the static site.
The first step in making a Webmention is to express that intent in our
Org Mode source files. indie-org
implements this by
See Adding Hyperlink
Types in Adding Hyperlink Types, Adding Hyperlink Types.
When the site author invokes
indie-org-webmentions-enable
four new link types will be
defined:
Each link type has a few properties, in particular :follow
and
:export
. indie-org
’s link types all implement follow in
the same way as standard HTML links; it’s the export aspect in which
they differ. While they will all render as standard HTML, they will
take the opportunity to note each of them via
indie-org-webmentions-record-webmention
For example, the author can indicate a mention like so:
I am mentioning Alice's [[mention:https://alice.net/i-made-a-thing.html][post]].
The site author shall include their publication state’s
indie-org-webmentions-made
(see Publication Environments & State) in the project properties (under the name
:indie-org/webmentions-made
); at export time, the custom link
types will, first, produce See Microformats appropriate to the sort
of webmention being made, then invoke
indie-org-webmentions-record-webmention
to record them in the
current publication environment’s state:
(defun iosh/publish (prod) "Publish indie-org.sh to production if PROD is non-nil, locally else." (indie-org-enable) ;; ... (let* ((env (if prod :prod :local)) (pub-states (if (file-exists-p publication-state-file) (indie-org-state-read publication-state-file) `(:prod ,(indie-org-state-make) :local ,(indie-org-state-make)))) (publication-state (plist-get pub-states env)) (webmentions-made (or (indie-org-state-v2-webmentions-made publication-state) (indie-org-webmentions-make-made))) ;; ... ) ;; This is all to setup the call to `org-publish-all'. (let* ( ;; ... (org-publish-project-alist `(("indie-org.sh" :components ("h-feed" "pages" "posts" "rss")) ("posts" ;; ... :indie-org/webmentions-made ,webmentions-made)))) (org-publish-all t) ;; During the course of publication, we may have updated state: in ;; particular, we may have made new webmentions & POSSE requests. Update ;; the publication state's most recent publication time... (setf (indie-org-state-v2-last-published publication-state) (current-time) ;; along with new requests: (indie-org-state-v2-webmentions-made publication-state) webmentions-made) (plist-put pub-states env publication-state) ;; and write it all out to disk. (indie-org-state-write pub-states publication-state-file))))
You can see an example on indie-org.sh here.
At this point, assuming the site author has made & recorded one or more webmentions (see made & recorded), they presumably want to send them.
Despite the protocol’s simplicity, sending Webmentions is a non-trivial task. The sender needs to carry-out Webmention endpoint discovery on their targets (which itself involves fetching & parsing the resulting document). Then there’s the question of handling failures to send & re-trying. Finally, since Webmention processing is generally done asynchronously, the sender has to “check back” to get the final result.
Aaron Precki has built a service to handle this: Telegraph. Once you sign-up (using See IndieAuth, of course), you can send a Webmention with a single API call over HTTP.
indie-org
simplifies this further with
indie-org-webmentions-send
a function that takes a source,
target, and Telegraph API token and will send the Webmention (recall
that the precise sort of Webmention is not encoded in the protocol;
the receiver discovers it by parsing the sender’s post).
We can record sent Webmentions in
indie-org-webmentions-sent
, another attribute of
indie-org-publication-state
. The process of taking both &
determining what Webmentions are to be sent is encapsulated in
indie-org-webmentions-required
.
So equipped, we can now code-up a function in our site’s Lisp that we can call post-publication that will handle sending all new Webmentions:
(defun iosh/send-webmentions (prod) "Send webmentions post-publication. PROD shall be set to t for production and nil for staging." (indie-org-enable) (let* ((env (if prod :prod :staging)) (all-pub-states (if (file-exists-p publication-state-file) (indie-org-state-read publication-state-file))) (publication-state (or (plist-get all-pub-states env) (indie-org-state-make))) (webmentions-made (or (indie-org-state-v2-webmentions-made publication-state) (indie-org-webmentions-make-made))) (webmentions-sent (or (indie-org-state-v2-webmentions-sent publication-state) (indie-org-webmentions-make-sent))) (webmentions-to-send (indie-org-webmentions-required webmentions-made webmentions-sent)) (token (indie-org-read-token telegraph-io-token-file)) (now (current-time))) (while webmentions-to-send (let* ((wm (car webmentions-to-send)) (location (if prod ;; The source is just the page key ;; (e.g. "blog/foo.html")-- it is only here that we ;; know we're sending this for prod, so prepend the ;; "https://indie-org.sh" here: (let ((src (car wm)) (dst (cdr wm))) (indie-org-webmentions-send (cons (concat "https://indie-org.sh" src) dst) token)) (message "Will send webmention: %s :=> %s" (car wm) (cdr wm)) nil))) (indie-org-webmentions-record-sent wm now webmentions-sent location)) (setq webmentions-to-send (cdr webmentions-to-send))) (setf (indie-org-state-v2-webmentions-sent publication-state) webmentions-sent) (plist-put all-pub-states env publication-state) (indie-org-state-write all-pub-states publication-state-file)))
Receiving Webmentions may seem like a particularly daunting problem for the staticly generated site owner. In order to receive them, we need a server capable of handling the HTTP posts and implement the server-side of the protocol (Webmention validation, asynchronous processing, possibly another endpoint at which senders can check their results).
It turns out, there’s a service for that: webmention.io. Once you sign-up via See IndieAuth, advertise your Webmention endpoint like so:
<link rel="webmention" href="https://webmention.io/indie-org.sh/webmention" /> <link rel="pingback" href="https://webmention.io/indie-org.sh/xmlrpc" />
Webmention senders will now POST
to webmention.io who will store
them on your behalf. You can retrieve them via API call to webmention.io.
indie-org
handles this for you via
indie-org-webmentions-check
. This function takes the domain name
for which you are checking Webmentions, your webmtion.io API token, and
an
indie-org-webmentions-received
struct (another attribute of
indie-org-publication-state
). indie-org-webmentions-check
will update the struct with any new Webmentions made.
So armed, we can add a method to our site’s Lisp, meant to be invoked periodically, for checking Webmentions made to our site:
(defun iosh/check-webmentions () "Check for new webmentions; update publication state." (indie-org-enable) (let* ((env :prod) ;; no staging support ATM (publication-state (if (file-exists-p publication-state-file) (indie-org-state-read publication-state-file))) (state-for-env (or (plist-get publication-state env) (indie-org-state-make))) (webmentions-received (or (indie-org-state-v2-webmentions-received state-for-env) (indie-org-webmentions-make-received))) (token (indie-org-read-token webmentions-io-token-file))) (indie-org-webmentions-check "indie-org.sh" token webmentions-received) (setf (indie-org-state-v2-webmentions-received state-for-env) webmentions-received) (plist-put publication-state env state-for-env) (indie-org-state-write publication-state publication-state-file)))
The site owner will need to arrange for this method to be invoked on a regular basis, perhaps as a cron job. If the owner wishes to display mentions as part of their page template(s), they will need to re-publish on receipt of new Webmentions.
Adding See POSSE to one’s site is a bit like See Sending Webmentions; simple in principle, but once one begins digging into the details it can quickly become daunting. Also like sending Webmentions, there’s a service that takes this off our hands: brid.gy.
brid.gy supports publishing:
as well as a growing list of sites.
This subsection will describe the process of adding POSSE to an Org Export-generated site, using indie-org.sh as an example. It will focus on Mastodon.
The first step is to sign-up at brid.gy. There are extensive docs at https://brid.gy/about. When you land on the home page, you’ll be asked to click on a tile representing the account to which you want to enable POSSE. On clicking “Mastodon”, you’ll be asked whether you want to post to a Mastodon account or connect to the fediverse. Select the former. You’ll next be asked for the address of your Mastodon instance, where you’ll have to sign-in. Once you’ve authenticated there, you’ll be asked for the address of your site– that’s it.
As an aside, brid.gy works via See Webmentions, both for accepting POSSE requests and for backfeeding responses to your POSSE’d content, so you’ll need Webmention support, first.
indie-org
integrates POSSE support into Org Mode, first by
defining a new keyword, #+POSSE
. When authoring one’s document,
one expresses a desire for POSSE by setting it like so:
#+TITLE: My Post #+AUTHOR: me #+POSSE: mastodon flickr ...
During the See Publication Process, indie-org
will note the POSSE
requests & record them in the publication state for future reference
(Cf.
indie-org-posse-requests
).
indie-org
also handles sending the requests and recording
the responses in
indie-org-send-posse-request
. The response can be recorded
with the publication state in
indie-org-record-sent-posse
. Given an
indie-org-posse-requests
instance and and instance of
indie-org-posse-responses
, the outstanding requests (i.e. the
POSSE requests that still need to be sent) may be found with
indie-org-posse-required
.
So equipped, we can give ourselves another function:
(defun iosh/send-posse-requests (prod) "Send POSSE requests post-publication. PROD shall be t to select production & nil to select staging." (message "Sending POSSE requests (if any)...") (indie-org-enable) (let* ((env (if prod :prod :local)) (all-pub-states (if (file-exists-p publication-state-file) (indie-org-state-read publication-state-file))) (publication-state (or (plist-get all-pub-states env) (indie-org-state-v2-make-publication-state))) (requests (or (indie-org-state-v2-posse-requests publication-state) (indie-org-posse-make-requests))) (responses (or (indie-org-state-v2-posse-responses publication-state) (indie-org-posse-make-responses))) (to-send (indie-org-posse-required requests responses))) ;; `to-send' will be a list of cons cells, each of whose car is a ;; page key & whose cdr is a list of POSSE symbols. (while to-send (let* ((page-key (caar to-send)) (source (concat "https://indie-org.sh/" page-key)) (symbols (cdar to-send))) (while symbols (if prod (let ((rsp (indie-org-send-posse-request source (car symbols)))) (indie-org-record-sent-posse page-key rsp responses)) (message "Will send POSSE request: %s :=> %s" source (car symbols))) (setq symbols (cdr symbols)))) (setq to-send (cdr to-send))) (setf (indie-org-state-v2-posse-responses publication-state) responses) (plist-put all-pub-states env publication-state) (indie-org-state-write all-pub-states publication-state-file)) (message "Sending POSSE requests (if any)...done."))
brid.gy automatically polls whatever silo you’ve connected to your site, looking for likes, responses, reposts, and so forth. It will then relay that information back to your site through... See Webmentions.
As an example, the author went to the Mastodon post POSSE’d from What Is indie-org.sh? and favorited it. After the next poll, a Webmention arrived representing that like and is now displayed here.
When POSSE’ing to space-limited fora such as Twitter & Mastodon, it can be inconvenient to have a lengthy post truncated by the target platform. brid.gy offers the ability to specify platform-specific text to be sent to the target silo.
We can take advantage of the Org Export back-end feature
:filters-alist
to make this more
convenient. :filters-alist
is a property list defining various
filters to be applied to the Org document during publication. Salient
to our purposes is :filter-parse-tree
: this filter is applied
to the Org document’s abstract syntax tree after the initial parse.
indie-org.sh
sets this filter to the function
iosh/post/filter-parse-tree
; it will search for a top-level Org
heading tagged with toot-text
. If found, that heading will be
removed from the parse tree, but the contents will be used when
POSSE’ing to Twitter & Mastodon.
For instance:
#+TITLE: My Post #+DESCRIPTION: I made a thing ... * Introduction This is a very long post.... * The First Thing ... blah. Blahblahblah. * Mastodon :toot-text: I wrote a very long post!
When this document is POSSE’d to Mastodon, only the text “I wrote a very long post!” will appear, along with a link to the original document.
Both the indie-org
package and the demonstration site
https://indie-org.sh are early code, and the author has tried to
indicate that with the 0.x
version number for the package.
In order to use indie-org
successfully, the site author is
going to need to be familiar with the Org Export facility
(see Org Export in Org Export) and
comfortable with See Emacs Lisp in Emacs
Lisp. That said, if one is hosting an Org Mode staticly-generated
site, those requirements are likely already met.
All that said, the recent meltdown at Twitter, shutting-down API access to numerous applications including brid.gy, only reinforces the imperitive to take control of our own identity, data & platform.
Furthermore, indie-org
is not just an experiment: I’m using it
today with my personal site.
The author gave a talk on one small part of this at EmacsConf. The documentation is hosted on-line here.
Bug reports & feature requests are welcome at sp1ff@pobox.com.
Jump to: | I L M O P W |
---|
Jump to: | I L M O P W |
---|
Jump to: | I O |
---|
Jump to: | I O |
---|