Build1 publisher3 min readPublished
LectuLibre carries a glossary across chunks to stop Elara becoming Elena by chapter 20
LectuLibre's dev.to write-up splits an EPUB on its own chapters and paragraphs, caps each chunk near 4,000 tokens, and feeds a glossary of proper nouns extracted from earlier chunks back into every translation prompt.
The Engineer · Build desk
What happened
- LectuLibre says it first translated independent chunks sized to the model's context window, and abandoned that approach for books running past 100,000 words.
- The pipeline now pulls chapters out of an EPUB with ebooklib and BeautifulSoup, then splits them at paragraph boundaries into chunks of roughly 4,000 tokens counted by tiktoken.
- Each new chunk opens with the tail of the previous one, prepended before the paragraph that overflowed the budget and counted against that budget.
- After each chunk is translated, spaCy pulls out entities, the LLM is asked to verify a consistent rendering, and the resulting dictionary goes into the system prompt.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost Whoever adopts this pays for consistency on every chunk, not once per book: an extra verification call alongside each translation call, plus budget spent re-sending the previous chunk's tail.
- constraint The glossary is built forward from finished translations, so chapter 1's rendering of a name governs chapter 20, and fixing a bad first choice means re-running work already paid for.
- exposure Copy the snippet as published and the overlap you get is measured in words, while the chunk cap beside it is measured in tiktoken tokens from a single encoding.
- decision A team sizing a long-document job has to decide separately how names stay stable, because buying a larger context window does not populate a terminology record.
Extraction runs on the translation of a chunk, not on the source text ahead of it [18]. The dictionary handed to chunk N therefore holds only the entities that surfaced in chunks 1 through N-1 [19]. The first appearance of a name fixes its spelling for everything after it. If chunk 1 renders it wrong, the mistake goes to every later prompt as instruction.
The overlap does not do that job. LectuLibre's post says so directly: "Overlap alone doesn't solve terminology consistency." [16] Its example is a character introduced in chapter 1 as "Elara" who is "Elena" by chapter 20, because the model does not remember the earlier choice [17].
The overlap default sits in the function signature, `chunk_chapter(chapter_text, max_tokens=4000, overlap_tokens=200)` [10]. The line that builds it is `' '.join(current_chunk.split()[-overlap_tokens:])` [11]. `str.split()` returns whitespace-delimited words, so the default carries the previous chunk's last 200 words, and the sizes the post compared were word counts too [12]. "100 tokens was sometimes not enough for a sentence to complete; 500 tokens added cost without quality gain," the write-up said [13]. A parameter named in one unit and spent in another survives because nothing crashes.
The overlap text is prepended to the new chunk and the token count recomputed from the combined string, so it spends part of the 4,000 [14]. 200 of 4,000 is 5 percent of each chunk given over to text the model has already read [15]. The entity verification step adds a second model call per chunk, on top of the translation call [20].
For 4,000 and 200 to transfer to another corpus, several things have to hold. Chapters have to arrive as `<p>` elements, since extraction pulls paragraph text out of the EPUB with BeautifulSoup and joins it with blank lines, and the chunker splits on those blank lines [7][8]. The receiving model's tokenizer has to track cl100k_base closely, because that one encoding does all the counting and the comment beside it claims only that it "Works for many models" [9]. And the quality bar has to be LectuLibre's: it calls 4,000 tokens "a sweet spot we found for quality vs. cost" and reports no measured comparison behind either number [8][23].
One branch fires only when a single paragraph exceeds `max_tokens`, which the code comments call rare for normal prose [21]. It splits that paragraph on `'. '` and rejoins the pieces with a single space. The delimiter is consumed, so every piece but the last loses its period [22].
The post's reason for not simply using a bigger window is that Claude's 200k tokens and GPT-4 Turbo's 128k still fall short of a full novel, and that cost and quality degrade with massive inputs [5]. What broke in the naive version, by its own account, was the naming: "names changed spelling halfway through, terminology was inconsistent, and dialogue references to earlier events sometimes made no sense" [4].
What to watch
- The available text breaks off mid-sentence while describing prompt injection; the full post may say whether the glossary is ever re-applied to chapters already translated.
- A measured comparison of overlap sizes on a fixed corpus would show whether 200 transfers beyond LectuLibre's books.
- Whether the token budget gets recomputed with each target model's own tokenizer instead of cl100k_base for everything.