Emacs Org Mode is a fascinating piece of software. At one level, it's another markup language. By which I mean a language in which one describes the structure and formatting of a document in the abstract, leaving it to a publication process to produce the document for final display purposes. Much like Markdown, one can attach metadata to the document, describe headings or chapters, indicate text variants (bold, italics, strikethrough or fixed-width, e.g.) and so on. Here's an example:

#+TITLE: Example Document
#+DATE: <2024-03-18 Mon 20:38>

* Heading One

We can *bold* text, _italicize_ it, +strike it out+ or mark it as =fixed width=.
** This Is A Sub-Heading

We can express lists:

  - with multiple
    + levels
  - [ ] and even indicate "todos"
  - [-] that are in-progress
  - [X] or done

or include tables

|                  |  a | b             |  c | d |
|------------------+----+---------------+----+---|
| [2024-03-16 Sat] | 32 | hello, world! | 64 |   |

or links: The Org Manual... and much, much more.

Beyond merely being a markup language, however, Org Mode is implemented in Emacs, enabling a computational layer on top of the markup. We can, for instance, add formulas to our tables:

| date             | total |
|------------------+-------|
| [2024-03-14 Thu] |     3 |
| [2024-03-15 Fri] |     4 |
| [2024-03-16 Sat] |     2 |
|------------------+-------|
|                  |     9 |
#+TBLFM: @>$2=vsum(@<<..@>>)

or add a little Emacs Lisp for keeping the DATE attribute up-to-date automatically:

(add-to-list 'org-export-options-alist
             '(:autodate "AUTODATE" nil "nil" parse))

(defun sp1ff/org/maybe-update-date ()
  "Update the DATE export keyword, if AUTODATE is non-nil."

  (interactive)
  (when (eq major-mode 'org-mode)
    (let ((auto-date
           (intern
            (org-no-properties
             (org-element-interpret-data
              (plist-get (org-export-get-environment) :autodate))))))
      (if auto-date
          (save-excursion
            (goto-char (point-min))
            (if (re-search-forward "^#\\+DATE:[         ]+\\(.*\\)" nil t)
                (replace-match (format "#+DATE: %s" (format-time-string "<%Y-%m-%d %a %H:%M>")))
              (progn
                (goto-char (point-min))
                ;; We are at the beginning of the buffer; skip one line ahead as long as we're looking at "#+..."
                (while (looking-at "^#\\+")
                  (forward-line))
                (if (bolp)
                    (insert (format "#+DATE: %s\n" (format-time-string "<%Y-%m-%d %a %H:%M>")))
                  (insert (format "\n#+DATE: %s" (format-time-string "<%Y-%m-%d %a %H:%M>")))))))))))

(add-hook 'before-save-hook 'sp1ff/org/maybe-update-date)

Now, if we add the AUTODATE attribute and set it to t, the DATE will be updated on save:

#+TITLE: My Document
#+DATE: <2024-03-18 Mon 20:39>
#+AUTODATE: t

Text here...

Most intriguingly we can embed arbitrary code into an Org Mode document, evaluate it, and pass the results of that evaluation to other source code blocks, perhaps in other languages, making Org Mode documents "live" workbooks:

#+NAME: example-table
| 1 |
| 2 |
| 3 |
| 4 |

#+NAME: table-length
#+BEGIN_SRC emacs-lisp :var table=example-table
  (length table)
#+END_SRC

#+RESULTS: table-length
: 4

#+BEGIN_SRC bash :var len=table-length :results output
  len=$((2*$len))
  echo "Twice the table length is $len."
#+END_SRC

#+RESULTS:
: Twice the table length is 8.

This is just the tip of the iceberg, but salient to this post is the Org Mode export feature to which I alluded above: one may export an Org Mode document to other formats, such as plain text ASCII, Beamer, HTML, LaTeX, Markdown, OpenDocument & Texinfo.

\(\TeX\) was one of the earliest markup languages. It was written by the legendary Donald Knuth in response to his disappointment at the galleys for the second volume of his classic The Art of Computer Programming. The "Hello, world!" TeX document is:

Hello, world!
\bye               % marks the end of the file

TeX will process this file to produce a .dvi file ("dvi" for DeVice Independent). The dvi file can be viewed directly or converted to other formats, PDF being the most common today. Despite dating back to the seventies, it remains the standard for document preparation in academic fields. Of particular note here is the fact that it provides a macro language which has been used to build more specific systems, such as \(\LaTeX\).

LaTeX was originally written by Leslie Lamport in the eighties, and provides authors with higher-level commands, implemented in terms of TeX macros, for salient aspects of authoring articles; things such as chapters, graphics, cross-references, bibliographies & many, many more. It is this TeX variant that is the standard for writing scholarly articles in mathematics, computer science and (as I understand it) the hard sciences altogether.

One aspect of both TeX & LaTeX that is germane to our purposes is that they both employ a distinct "mode" for typesetting mathematical expressions. One may enter math mode in a few ways:

• inline math mode is used for expressing mathematics interspersed with text, e.g.

  If $x \ge 3$ then we may conclude that \(y \in \Gamma\).
  

  will typeset to "If \(x \ge 3\) then we may conclude that \(y \in \Gamma\)." One marks stretches of text as being in inline math mode by surrounding them with either '$' characters or '\(' & '\)' (while researching this article, I came across an anecdote that Knuth chose the '$' symbol because math was, at the time, considered expensive to typeset)

• display math mode is used to express equations:

  will display the equation centered on its own line. Historically, '$$' was also used to denote this, but that seems to be deprecated.

• the equation environment: LaTeX environments are entered with \begin{name} and exited with \end{name} (where "name" identifies the environment being used). Environments are used to apply certain formatting conventions within that portion of the document. The "equation" environment, among other things, enters math mode.

The "Hello, world!" \(\LaTeX\) document is:

which will render as:

Paraphrasing the Org Mode manual, we can freely mix LaTeX math fragments into the document. The following stretches of text will be identified as LaTeX:

• "Environments of any kind. The only requirement is that the '\begin' statement appears on a new line, preceded by only whitespace"
• text marked by LaTeX math mode delimiters; '$', '\(' / '\)' & '\[' / '\]' (though they recommend against '$')

The interesting question is: what happens on export? For my particular case, only LaTeX & HTML export are of interest. In the latter case, the LaTeX snippets are passed through as-is… which is dangerous. Between Org Mode's willingness to accept arbitrary LaTeX environments and adding LaTeX packages via #+latex_header, I was able to do more & more in LaTeX while successfully exporting to LaTeX (and from there to PDF). In particular, I was able to reproduce Ukkonen's diagram like so:

#+LATEX_HEADER: \usepackage{tikz}
#+LATEX_HEADER: \usetikzlibrary{tikzmark}
# other headers...

text here...

\begin{table}[htbp]
\caption{minimal cost paths}
\centering
\begin{tabular}{lrrrrrrr}
 &  & p & a & p & l & e & t\\[0pt]
\hline
  & 0\tikzmark{a} & 2\tikzmark{b} & 4 & 6 & 8 & 10 & 12\\[0pt]
a & 2\tikzmark{c} & 3 & 2\tikzmark{d} & 4 & 6 & 8 & 10\\[0pt]
p & 4 & 2\tikzmark{e} & 4\tikzmark{f} & 2\tikzmark{g} & 4 & 6 & 8\\[0pt]
p & 6 & 4 & 5 & 4\tikzmark{h} & 5 & 7 & 9\\[0pt]
l & 8 & 6 & 7 & 6 & 4\tikzmark{i} & 6 & 7\\[0pt]
e & 10 & 8 & 6 & 8 & 6 & 4\tikzmark{j} & 6\tikzmark{k}\\[0pt]
\end{tabular}
\end{table}

\begin{tikzpicture}[remember picture, overlay]
\draw[->] ([xshift=0.5ex, yshift=0.75ex] pic cs:a) -- ([xshift=-1.5ex, yshift=0.75ex] pic cs:b);   % horiz
\draw[->] ([xshift=-0.5ex, yshift=0.0ex] pic cs:a) -- ([xshift=-0.5ex, yshift=1.75ex] pic cs:c);   % vert
... % more arrows here
\end{tikzpicture}

...text continues

tikz is a LaTeX package for diagrams. By importing it in my document's LaTeX headers, and then just writing the LaTeX in my Org Mode, the LaTeX was passed through unchanged when exporting to LaTeX and I got a very nice reproduction of Ukkonen's diagram:

Then I tried exporting to HTML:

Whoops. So, how does HTML export handle embedded LaTeX? It can do so in two ways: it can use MathJax at render time, or it can process the LaTeX at export time first to DVI, and then to an image file which will be included in the HTML.

All that said, there's a second way to embed LaTeX into Org Mode diagrams– setup a source code block (see above) containing LaTeX. But that brings up the question: what does it mean to "evaluate" a LaTeX code block? Well, per LaTeX Source Code Blocks in Org Mode: "LaTeX source code blocks can be used to tangle a LaTeX source file, or to create bitmap images or pdf snippets of arbitrary LaTeX code."

"Great!" I thought: I'll prefer LaTeX for my diagraming language & keep the diagram together with the document. Also, the ability to fine-tune the evaluation environment appealed to me.

#+BEGIN_SRC latex :results output file link replace :file ukkonen-minimal-cost-paths.png :buffer no :headers '("\\usepackage{tikz}" "\\usetikzlibrary{tikzmark}")
  \begin{table}[htbp]
  \caption{minimal cost paths}
  \centering
  \begin{tabular}{lrrrrrrr}
   &  & p & a & p & l & e & t\\[0pt]
  \hline
    & 0\tikzmark{a} & 2\tikzmark{b} & 4 & 6 & 8 & 10 & 12\\[0pt]
  a & 2\tikzmark{c} & 3 & 2\tikzmark{d} & 4 & 6 & 8 & 10\\[0pt]
  p & 4 & 2\tikzmark{e} & 4\tikzmark{f} & 2\tikzmark{g} & 4 & 6 & 8\\[0pt]
  p & 6 & 4 & 5 & 4\tikzmark{h} & 5 & 7 & 9\\[0pt]
  l & 8 & 6 & 7 & 6 & 4\tikzmark{i} & 6 & 7\\[0pt]
  e & 10 & 8 & 6 & 8 & 6 & 4\tikzmark{j} & 6\tikzmark{k}\\[0pt]
  \end{tabular}
  \end{table}

  \begin{tikzpicture}[remember picture, overlay]
  \draw[->] ([xshift=0.5ex, yshift=0.75ex] pic cs:a) -- ([xshift=-1.5ex, yshift=0.75ex] pic cs:b);   % horiz
  \draw[->] ([xshift=-0.5ex, yshift=0.0ex] pic cs:a) -- ([xshift=-0.5ex, yshift=1.75ex] pic cs:c);   % vert
  % ... many more
  \end{tikzpicture}
#+END_SRC

Huh:

the reader will note that my carefully-drawn lines are nowhere to be found.

This took me some time to run-down, so I want to note it here for anyone else who stumbles upon this through search. It turns out that we often have to run LaTeX more than once. In particular, we need to do so when certain elements of the document require knowledge of the positions of other elements that are only known after a prior pass. This conversation on the Org Mode mail list put me onto the right path: and indeed, if we examine the variable org-latex-pdf-process:

org-latex-pdf-process is a variable defined in ‘ox-latex.el’.

Its value is
("%latex -interaction nonstopmode -output-directory %o %f"
 "%latex -interaction nonstopmode -output-directory %o %f"
 "%latex -interaction nonstopmode -output-directory %o %f")

Its docstring explains: "The reason why this is a list is that it usually takes several runs of ‘pdflatex’, maybe mixed with a call to ‘bibtex’. Org does not have a clever mechanism to detect which of these commands have to be run to get to a stable result, and it also does not do any error checking. Consider a smart LaTeX compiler such as 'texi2dvi' or 'latexmk', which calls the 'correct' combinations of auxiliary programs."

Ah. Alright, so what exactly was happening when I evaluated that source code block? Well, perusing the source turned up org-babel-execute:latex, which is documented as "executing" a LaTeX body. Edebug confirmed I was in the right place. It delegates to org-create-formula-image, which builds the image file in two steps: first by calling the LaTeX compiler, and then calling the image converter on the resulting dvi file. The details of what, exactly, comprises the "LaTeX compiler" are governed by the variable org-preview-latex-process-alist, where we see that by default, for each supported image type, the LaTeX compiler attribute is a list of strings containing a single invocation of LaTeX:

(defcustom org-preview-latex-process-alist
  '((dvipng
     :programs ("latex" "dvipng")
     ;; ...
     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f"))
    (dvisvgm
     ;; ...
     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f"))
    (imagemagick
     ;; ...
     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f"))
    ;; ...
    )

A bit of Emacs Lisp put things right:

(plist-put
 (cdr (assoc 'dvipng org-preview-latex-process-alist))
 :latex-compiler '("latex -interaction nonstopmode -output-directory %o %f"
                   "latex -interaction nonstopmode -output-directory %o %f"))

Given the fact that Org Mode correctly handles this in org-latex-pdf-process, I plan on submitting a patch for this.

Still, it seemed a bit of a shame to resort to including an image in the PDF output just because that's what HTML required. And what about an .svg file, to provide better scaling? The Worg page on LaTeX source code blogs contains a tantalizing example of backend-dependent export logic. It didn't come close to working out of the box, but I was intrigued enough to keep digging.

The idea is to compute different export headers for the LaTeX source code block, at export time, for each backend. For export to LaTeX, we setup headers that will simply pass through the LaTeX source "as is", whereas for HTML export we'll produce an .svg file and include a link to that file in the final HTML output.

In order to determine the "current" backend during the export process, we need to consult the variable org-export-current-backend; that's a little prolix, so let's give ourselves a macro that will allow us to express ourselves more succinctly (I placed it at the top of the document, in a comment):

* COMMENT setup

#+BEGIN_SRC emacs-lisp :results silent
  (require 'cl-lib)
  (defmacro by-backend (&rest body)
    `(cl-case org-export-current-backend ,@body))
#+END_SRC

I then updated my source block to this:

#+HEADER: :results (by-backend (latex "latex") (t "output file link replace"))
#+HEADER: :file (by-backend (latex 'nil) (t "ukkonen-minimal-cost-paths.svg"))
#+BEGIN_SRC latex :buffer no  :headers '("\\usepackage{tikz}" "\\usetikzlibrary{tikzmark}")
  \begin{table}[htbp]
  % ...
#+END$_SRC

So when exporting to LaTeX, the LaTeX source will be passed through unchanged. When exporting to any other backend, we'll "evaluate" the source block to produce an .svg & just include the image in the output document. I have no idea if this will work with backends other than HTML; I presume it will depend on the implementation details of the given backend. I selected export to HTML and…

Huh. Worse, it turns out that the code path for creating images from LaTeX for HTML export is completely different than that for "evaluating" LaTeX source blocks (with which I was already familiar, see above). I was able to track-down org-latex-compile due to a fortuitous log message & noticed that the LaTeX compilation step was choking on the line \def\pgfsysdriver{pgfsys-tex4ht.def} in the generated LaTeX file. An Org Mode list bug report clued me in to the variable org-babel-latex-preamble– I just updated it to exclude the offending line:

(setq
 org-babel-latex-preamble
  (lambda (_)
    "\\documentclass[preview]{standalone}
"))

et voilà: