Reproducible research
The publication ships with the code that produced it, and a reader can re-run the analysis. Stated at its most pointed by Buckheit and Donoho in 1995, quoted in org-mode-literate-programming:
"An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and complete set of instructions which generated the figures."
The compendium
Gentleman and Temple Lang (2004) gave the container a name — the compendium — holding text, code and data together, and proposed it as "the new unit of peer review and distribution of scientific work." Their list of what that buys, as org-mode-literate-programming reproduces it: the reader gets the author's actual work rather than an abridged version; derived documents can show different levels of detail; computations can be re-run with different inputs; plots and tables are constructed programmatically; and the components can be treated as data and manipulated in ways the author never planned for.
That last item makes the published artifact machine-readable by intent, twenty years before open-knowledge-format made the same move for notes.
Not the same thing as literate programming
The two traditions are usually collapsed and org-mode-literate-programming keeps them apart, on a directional test: literate-programming tangles code out of the document into a file the compiler reads; reproducible research runs code in, as part of generating the document. One serves the future maintainer, the other serves the sceptical reader. A tool can do either without the other, and until 2012 nearly every tool did exactly that — noweb and cweb on one side, Sweave on the other.
Sweave (Leisch 2002) is the tradition's centre of gravity: R blocks inside LaTeX, executed to produce a LaTeX document with the tables, figures and inline results filled in. It spawned SASweave, Statweave and odfWeave, reaching SAS, Open Document Format and Word. The paper credits Sweave and the R community with inspiring the compendium idea and with the revival of interest in reproducibility generally — and notes that none of them can reorder code blocks, so none is fully literate.
Where the corpus stands on it
Thin, and older than it looks. The spoke's live sources argue for documents that run — executable-markdown, wiki-compiler, the llm-wiki pattern — largely without reference to a tradition that had peer-reviewed tooling, a named unit of exchange, and a journal willing to publish the artifact in 2012. Whatever is new about LLM-written notebooks, the container is not.
Open
- No source here says whether the compendium proposal went anywhere: whether any journal adopted it as a review unit, or what share of computational papers ship runnable code now.
- The trust problem is the flip side of the goal. Code that a reader can re-run is code that runs on the reader's machine — see the safety section of org-mode-literate-programming and executable-markdown.
Related
literate-programming · org-mode-literate-programming · org-mode · executable-markdown · open-knowledge-format · tools-for-thought