Writing Setup
Published on 2024-11-13
My writing setup mainly revolves around two things: plain text, and completely local. Code is in plain text files, and there are a glut of tools for dealing with text (linters, regular expressions, etc.). Keeping all my writing in plain text means I’m never locked into a specific vendor. Any editor on any device and operating system under the sun can open a text file.
Keeping the source in plain text feels natural for a programmer. Of course the source of truth is text files that get compiled into something easier for others to read.
My setup for writing any text, whether I’m taking notes, writing an article with mathematics and citations, or creating a presentation, is adaptable enough to handle anything.
All writing begins with the input. My preferred text editor is Neovim. You can find my dotfiles here. Vim keybindings are burned into my brain at this point so it’s by far the fastest way to write and edit text for me. I write everything in Pandoc Markdown. It’s a slightly more powerful variant of Markdown that adds the ability to have metadata information, LaTeX math, and citations. Of course the format is tightly coupled with the Pandoc document converter. Markdown seems like the happy medium for syntax, where the most common cases require nearly no extra syntax, but you can always dip into raw LaTeX in those few circumstances where you need something more then what pure Markdown provides.
The next step in the process is the build phase. Everything is handled by a Makefile. All my Makefiles tend to follow the same pattern. I have some variables at the top containing my wildcard patterns matching the Markdown source files and the HTML or PDF files that I want to compile them to, and then a few rules to make each recipe. For example, here’s the Makefile that makes this blog.
.PHONY: all pack deploy clean test files
ifneq (,$(wildcard ./.env))
include .env
export
endif
POSTS=$(wildcard posts/*.md)
SPECIAL=$(wildcard special/*.md)
HTML_POSTS = $(patsubst posts/%.md,html/%.html,$(POSTS))
HTML_SPECIAL = $(patsubst special/%.md,special_html/%.html,$(SPECIAL))
all: files pack
files: $(HTML_POSTS) $(HTML_SPECIAL)
html/%.html: posts/%.md
pandoc --mathml --lua-filter filter.lua -o $@ $<
special_html/%.html: special/%.md
pandoc --mathml --lua-filter filter.lua -o $@ $<
pack:
rm -f blog.db
uv run pack.py
deploy:
cp blog.db ../jonathansm.com/
rsync -avzP blog.db $(DEPLOY_SERVER):$(DEPLOY_PATH)blog.db
clean:
rm -f html/*.html
rm -f special_html/*.html
rm -f blog.db
You can see flexibility of plain text at work here. I can use Lua filters in Pandoc and Python programs as apart of the build process.
The final important piece of my writing process is Zotero. It’s a database for anything that I want to cite. The Zotero Connector extension lets me file a website, article, or paper directly into Zotero with one click. The citations for each project that I work on are kept in their own folder. Whenever a new source is added or I change some metadata that folder is automatically exported in the background by the Better Bibtex plugin to a CSL YAML citation file. The citeproc extension in Pandoc then uses the citation information in the YAML file and style information from a csl file (I always use APA) to insert in-text citations and put the full works cited page at the end.
Currently I use a custom Lua command to call the Zotero API, let me search for the citation I want, and insert the correct reference into the text. I’ve looked at a writing a Neovim plugin that could interact with the Zotero app more directly so I could run a search over the API and display the search results in a native Neovim completion handler. Unfortunately the documentation seemed a little sparse when I last checked.
The final benefit of local plain text files is you can use the same version control tools you know and love. Good old Git is totally happy to store your Markdown files and keep track of every change you make.
One area I’ve been contemplating improving is adding a templating step to my build process. I’ve had a few times recently where separate documents needed to include a section of text and I had to duplicate the text between them. If I need to change the text they both need to include, the duplication means manually updating it in every document. A template engine would allow me to write the text twice and have it embedded into each document referencing it before it’s compiled, meaning updating the single piece of text would update it everywhere.