The lesson I would pass on to medievalists is this: in a text-recognition pipeline, the most consequential choices are made before any model is trained. The convention is the founding act, and writing it down is itself part of the scholarship.

Our own response was to align with an existing, semi-normalised standard (OpenITI / Dabīrān), to preserve alternative readings instead of forcing a single one, and to freeze the HTR test set only once the convention had stabilised. We document it and version it, like any other scholarly decision.

None of this is unique to my project; conventions are a running theme at The DO. Rachael Griffiths shows it in HTR for Tibetan, and it recurs well beyond OCR: TEI and cataloguing, standards for selmantic-web networks, metadata for Coptic resources, and interoperability in Indian-language archives.

Setting it means a run of concrete decisions: keep optional vowel pointing or not, expand scribal abbreviations or leave them, where marginal insertions join the line, one convention for Arabic and Ottoman Turkish or two. Each choice shapes what later analysis can see.

The other corpus I work on, the Jerusalem court registers, the sijillāt, has no machine-readable text at all. There the convention is not inherited; we have to set it ourselves. That turns out to be the founding act of the whole project rather than a technical detail.

This mattered to us because our two corpora sit at opposite ends. The Istanbul fatwā compilations have already been transliterated in printed editions, so we inherit someone else's convention, much as a medievalist works from a critical edition.

The comparison also surfaces real history. Two fatwā questions with near-identical wording drew opposite rulings a century apart, over whether a husband denying paternity must undergo li'ān. One jurist ruled that he must, the later one that he need not. It found that pair among tens of thousands.

The point generalises, and it should interest medievalists as much as us. Whatever you run downstream, whether similarity, distant reading, or named-entity recognition, still sits on top of how the text was transcribed. The convention you settled on decides, in advance, what you can detect.

The explanation turned out to be simple: the two printed editions had transliterated the Ottoman Turkish differently. Our method was comparing transcriptions, not the underlying texts. That missing 0.18 was the measured weight of a single editorial convention.

👋 Still the DO here for the takeover ! Here is a small puzzle from my work. We ran a text-similarity test on two Ottoman fatwā collections. Word for word, two copies of the same ruling should have scored 1.0, but came out at 0.82. The algorithm was fine, so the missing 0.18 needed explaining. 🧵

Today's 🧵 is all about 🐉📜!

That is the first entry in the bestiary. Tell us which creature we should set side by side next; suggestions are welcome below.👇

The possibilities offered through virtually consolidating materials are also explored in The DO e.g. Zhaleh Nayebossadrian’s piece on the Women’s Worlds in Qajar Iran project, Soni Wadhwa’s archive of Sindhi language advertisements, and Leon Woltermann’s essay on Indonesian short stories.

All three pages sit in the BnF and are digitised in Gallica, yet a medievalist and an islamicist would rarely open them in the same sitting. Only digitisation makes it possible to compare these distant pages at all.

Across the three pages, the dragon is evil, then a moral warning, then part of the cosmos. No single version is the real one; each tradition turns the creature to its own ends. The dates are worth noting too: a 15th-century French book set beside 18th-century Arabic copies.

The dragon is not always a threat, though. In this copy of al-Qazwīnī's Wonders of Creation the same creature becomes a constellation, al-Tinnīn, our Draco: a red, scaled serpent coiled across the night sky and read here as a map of the heavens.