In Praise of Documentation

Overview

How I learned to love documentation
The perils of poor documentation
Why document your code
Types of documentation
Literate programming
Scientific publishing tools
Computer programs as literature
Documentation in the age of AI

This presentation was made with ❤️ using Quarto, Python, and deployed with GitHub Actions.

How I learned to love documentation

The perils of poor documentation

Years later, I learned my early enjoyment for reading what we came to call “docs” was matched by an enjoyment in writing technical “docs”. But I quickly realised other data scientists and software engineers often didn’t share my interest or indeed passion for “the docs”. I often found people either wouldn’t document their code at all, or they would document it poorly, and only under duress. Bad things can happen in companies without good documentation. Documentation is the institutional memory of a company. Without (good) documentation, it becomes difficult to understand a codebase. New colleagues can feel alienated, struggle to quickly learn a new and complex codebase, or be unable to fix or update it. It might take a whole team of PhD-level developers months, even years, to reverse engineer a large legacy codebase, to be in a position to rebuild or replace it.

Why document your code

“Documentation is an often overlooked aspect of data science. It’s commonly left until the end of a project, but then you’re excited to move on to a new project, and the documentation is rushed or omitted completely. However, … documentation is a crucial part of making your code reproducible. If you want other people to use your code, or if you want to come back to your code in the future, it needs good documentation. It’s impossible to remember all your thoughts from when you originally wrote the code or initially carried out the experiments, so they need to be recorded.” (Nelson 2024)

Types of documentation

“Names: Names of variables, functions, and files should be informative, an appropriate length, and easy to read.

Comments: Your comments should add extra information not contained in the code, such as a summary or a caveat.

Docstrings: Your functions should always have a docstring that describes the inputs and outputs of the function, as well as the purpose of that function.

READMEs: Every repository or project should have an introduction that advertises your code and lets other people know why they should use it.

Experiment tracking: Experiments, especially in machine learning projects, should be tracked in a structured way.”

Jupyter notebooks: Your notebooks will be much easier to read if you give them good names, give them a structure, and intersperse text and code.

Nelson (2024)

Development loop

graph TD
A[Start feature or analysis] --> B[Explore & experiment]
B --> C[Write code]
C --> D[Write & update documentation]
D --> E[Review & refactor]
E --> F[Share / deploy]
F --> B

Literate programming

A document formatting language
A programming language

Knuth (1984)

Nelson’s advice, especially her advice for how to use Jupyter notebooks, can be considered advice for “literate programming”. Literate programming is perhaps the most commonly recommended genre or style of computer programming for data science. The term “literate programming” was coined in 1984, by the American computer scientist Donald Knuth, later Professor Emeritus at Stanford University (Knuth 1984). In literate programming, written prose is interspersed with computer code, within the same document, called a “notebook”. Knuth suggests literate programming is “inherently bilingual”, because it combines two different genres of writing within the same computer program files: a document formatting language and a programming language. While the basic technical implementation of literate programming will be familiar to most data scientists, the origins and history of literate programming might be less familar.

History of literate programming

WEB system:

A document formatting language (TeX)
A programming language (PASCAL)

A “tool for systems programmers, not for high school students or hobbyists”, because the programmer needs to be “comfortable dealing with multiple languages simultaneously” (Knuth 1984).

Google Ngram plot for literate programming (1984+, 5y smoothed)

Decade	Mean	Max	Max year	Years
Google Ngram: "literate programming"
Decade patterns (1984–2019), per-billion tokens
1980s	0.095	0.357	1989	6
1990s	0.320	0.659	1998	10
2000s	0.268	0.594	2000	10
2010s	0.216	0.970	2019	10

Google Ngram table for literate programming (1984+, 5y smoothed)

IPyLeaflet map of Cambridge, UK

Scientific publishing tools

Jupyter notebooks

“Notebooks — documents integrating prose, code and results — offer a way to publish a computational method which can be readily read and replicated.”

“Prose text can be interleaved with the code and output in a notebook to explain and highlight specific parts, forming a rich computational narrative.”

(Kluyver et al. 2016)

Limitations of Jupyter notebooks

Pimentel et al. (2019) analysed 1.4 million notebooks on GitHub and found that “out of 863,878 attempted executions of valid notebooks (i.e., notebooks with defined Python version and execution order), only 24.11% executed without errors and only 4.03% produced the same results.”

They make eight recommendations for improving the use of Jupyter notebooks.

Solutions to the limitations of Jupyter notebooks

Jupytext
Marimo notebooks
Nelson recommends using Jupyter notebooks for development work, before refactoring notebooks into modularised Python scripts with accompanying tests (Nelson 2024).

R Markdown and Quarto

Example of Quarto YAML

---
title: "In Praise of Documentation: Tools, Tips & Techniques for Literate Programming in the AI Age"
date: "2026-03-01"
categories: [documentation, talks]
format:
  html:
    toc: true
    code-fold: true
jupyter: pydata
bibliography: refs.bib
---

You render the Quarto document with quarto render talk.qmd on the command line.

Computer programs as literature

“I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature”

“Let us change our traditional attitude to the construction of progams. Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do”.

“The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other”.

Knuth (1984)

While Knuth’s literate programming style will be familiar to most data scientists, his original ideas about computer programming might be less known.

Knuth makes several arguments to support his “motto” of literate programming.

I was frankly astonished when I read these paragraphs. They are a very unusual, and bold, set of arguments for a computer scientist to make. Knuth appears to be flipping the disciplinary association of programming from computer science - whether based in mathematics or engineering (so-called “technical” disciplines or “hard” sciences) - to literature, a non-technical, humanities “soft” discipline. The technical domain of programming is turned into a literary domain.

I find it fascinating to think about how both data science and software engineering could be different, if computer software was considered as works of literature, and programmers as essayists.

Rules for good writing

George Orwell (Orwell 1945) considered good writing to be active, precise, and simple.

He argued we should write actively rather than passively, be precise rather than vague, and use simple verbs rather than complex longer words or phrases.

If we can drop a word from a sentence without losing the meaning, we should drop it.

Rules for good writing

Orwell called the use of language a habit. And he made a close relationship between language and thought.

By learning to write well, we can develop good ways of thinking. By writing clearly, we can think clearly, and thereby communicate our thoughts more clearly to others.

The thoughts are in the words, and our thinking is displayed in our writing, so we had better choose our words wisely.

Documentation in the age of AI

Spec-driven development
AGENTS.MD
Coding agent CLI or IDE tools can be instructed to read the AGENTS.md and do “spec-driven development” (e.g. Cursor, OpenAI Codex, Claude Code).

In the present day, software engineering is being transformed by AI, especially through code asssistants and agentic workflows, where we instruct agents to write our code and documentation for us. I’ve noticed developers are particularly keen to outsource the boring (or difficult) work of writing documentation to AI.

At the same time, written documentation is becoming more important for AI-assisted workflows. The following trends are becoming popular in software engineering, which will surely impact data science too:

Spec-driven development reverses the usual approach to documentation in software engineering, where documentation is written after the software itself. Instead, specifications are written in a machine-readable format, which can then be read by coding agents.

AGENTS.MD is an emerging standard way of implementing spec-driven development. /This is a text file in Markdown format, written to be read by coding agents, which they use to create a plan and then execute that plan.

Benefits of writing good documentation

Feel like a human being, rather than a machine
Feel good about ourselves, be appreciated by others, whilst also making the world a (slightly) better place
Preserve institutional and personal memory
Ensure scientific reproducibility
Allow better version control, so software is robust
Enable responsible stewardship of technical systems, to allow future maintenance
Documentation can be technically “fun”

Concluding remarks

Recommended Resources

To read a longer version of this talk, visit the blog @ carecodeconnect.io

Diátaxis: documentation framework

Write the Docs: best practices for creating software documentation and technical writing.

References

Gazzard, A. 2016. Now the Chips Are down: The BBC Micro. The MIT Press.

Kluyver, T., B. Ragan-Kelley, F. Pérez, B. Granger, M. Bussonnier, J. Frederic, K. Kelley, et al. 2016. “Jupyter Notebooks – a Publication Format for Reproducible Computational Workflows.” In Positioning and Power in Academic Publishing: Players, Agents and Agendas, edited by F. Loizides and B. Schmidt.

Knuth, D. E. 1984. “Literate Programming.” The Computer Journal 27 (2): 97–111. https://doi.org/10.1093/comjnl/27.2.97.

Nelson, C. 2024. Software Engineering for Data Scientists: From Notebooks to Scalable Systems. O’Reilly.

Orwell, G. 1945. Politics and the English Language. London: Penguin Books.

Pimentel, J. F., L. Murta, V. Braganholo, and J. Friere. 2019. “A Large-Scale Study about Quality and Reproducibility of Jupyter Notebooks.” In IEEE/ACM 16th International Conference on Mining Software Repositories, 507–17. Montreal, QC, Canada. https://doi.org/10.1109/MSR.2019.00077.