We Generated a 50-Chapter AI Novel. Here Is Everything That Broke.
An illustrative walkthrough of a 50-chapter consistency stress test: how to set it up, what tends to break at each stage, and what to log.
Writing with Language Models
Novarrium Team
This is an illustrative walkthrough of the kind of test we run, not a published study. The characters, chapter numbers and examples below show what to look for, not results you should expect. What the checks do today is in the 9 checks.
Writers keep asking us the same question: "Does this actually work past chapter 20?"
Fair question, and the honest answer is: test it on your own book. A language model drafting from a long history tends to slip character details somewhere past chapter 10, lose plot points as the book grows, and contradict itself more often the further it goes. This post walks through how to build a 50-chapter stress test and what to look for at each stage.
The Setup: Building a Complexity Stress Test
A simple "hero's journey" story will not find the weak spots. A useful test book is built for maximum complexity, for example:
- 12 named characters with distinct personalities, backstories, and physical descriptions
- 3 factions (a military order, a merchant guild, a religious sect) with shifting alliances
- A magic system with specific costs, limitations, and consequences
- 4 romantic subplots running parallel to the main conflict
- A 6-month timeline with specific dates for key events
- Multiple POV characters requiring consistent voice differentiation
Epic fantasy is a good genre for it because it is the most consistency-demanding. Get a character's eye color wrong in a romance novel and readers might forgive you. Resurrect a dead character in chapter 31 when they died dramatically in chapter 18? Your review section becomes a war zone.
Before chapter 1 is drafted, write down every detail a reader would notice if it changed: physical descriptions, relationships, faction decisions, magic rules, timeline markers. Better still, confirm them in the Story Bible, because a fact you wrote or confirmed is what a later chapter is held to.
Chapters 1-10: The Honeymoon Phase
Expect the first ten chapters to look smooth.
Why? The context window is not yet stressed. With only ten chapters of history, most of what matters still fits in what the model is shown, so even a tool with no fact store looks good here.
This is the zone where most writing tools look impressive. Demo videos showcase chapters 1-5 for a reason. What to do at this stage: review the facts the Story Bible proposes after each chapter and confirm the ones that matter. That work pays off later.
Chapters 11-20: Where Most AI Tools Fail
This is where the first near-misses typically show up. Three kinds to watch for, with made-up examples:
A trait slips toward a trope. Say Eirik is given "steel-gray eyes" in chapter 2, and a romantic scene in chapter 13 reaches for "piercing blue eyes", because the model is drawing on common romance phrasing rather than your character. If the eye colour is a fact you wrote or confirmed, Logic-Locking checks the drafted chapter against it, and a plain contradiction is repaired in place or the chapter is redrafted.
Timeline arithmetic. A character says "two weeks ago" about an event your timeline puts four weeks back. Check these by hand as well: date arithmetic is easy to get wrong and easy to miss.
Invented history. Two characters who have never met on the page "renew their old friendship". The model filled a gap with something plausible. A relationship call like this usually comes back as a note quoting the line, for you to decide.
This is the zone where most AI writing tools start breaking down. The story has more history than the model can be shown at once, and without a store of facts it fills the gaps with plausible-sounding but incorrect details.
Chapters 21-35: The Stress Zone
By now the Story Bible holds hundreds of facts, relationships have changed many times, and faction politics have layers of alliance and betrayal.
Find the contradictions before a reader does
Paste a chapter right here. The free Continuity Scan quotes every contradiction from both sides. No signup, 3 scans a day. Open the full scan page
Add your Story Bible (optional): the facts and rules to check against
Without it, the check uses what your text establishes earlier on.
Chapter 23: The Dead Character Problem
Say Thyra dies heroically in chapter 18, with an emotional funeral in chapter 19. A council meeting in chapter 23 is a classic place for her name to turn up in the list of attendees: the model has seen her name often and has no living-or-dead switch of its own.
What to check: is her death in the Story Bible, and did you confirm it? A death you wrote or confirmed is checked against every later chapter, and a chapter that brings her back is repaired or held. A death the system inferred and you never confirmed raises a question instead of a hold, so confirm the deaths that matter.
Chapter 28: Timeline Arithmetic Failure
"Three months since the siege" when two months have passed. Not a huge error, but these small slips accumulate: left alone, "three months" becomes "six months" by chapter 40.
Chapter 31: Faction Politics Complexity
A merchant guild that has switched sides twice makes a good trap. In a negotiation scene, does a guild member mention "our long partnership" with a faction they betrayed eight chapters earlier? Catching that needs relationship history, not just current relationships. Record the betrayal in the Story Bible when it happens; whether a line of dialogue contradicts it is often a judgment call, so expect a note rather than a hold.
Chapters 36-50: Full Scale Novel Complexity
By chapter 36, a book like this has serious complexity:
- Characters have 35+ chapters of development history
- Relationships have evolved through multiple stages
- Faction politics have three layers of depth
- Multiple subplots are converging toward the climax
- The magic system has been used in dozens of scenes with specific consequences
Chapter 42: Character Development Continuity
A character who spent 20 chapters growing from arrogant warrior to humble leader is described with their chapter 5 traits. This is a craft judgment, not a contradicted fact, because people do regress. Expect it to come back as a note, and decide whether the regression is earned.
Chapter 47: Magic System Rule Violation
Say fire magic causes temporary blindness. In chapter 47 a character uses fire magic and then describes what they see in the same scene. If that rule is one you set, the chapter is checked against it like a fact, and a chapter that breaks it is repaired, redrafted or held.
Chapter 50: Arc Resolution Consistency
The final chapter has to call back to events from chapter 3, character development from chapter 12 and a subplot introduced in chapter 8. Without a store of facts, endings tend to come out generic and disconnected. With one, the relevant facts are put in front of the model when it drafts. What to check: does the ending cite actual events from the book, or vague ones?
What Breaks: The Failure Log to Keep
The output of a test like this is a log. For each problem the checks catch, or you catch, record the chapter, the kind of problem and what happened to it. The kinds worth tracking:
- Physical description drift (eye color, hair length, scars, height)
- Dead character resurrection
- Timeline contradictions
- Relationship confusion
- Faction politics errors
- Magic system violations
- Character development regression
- Subplot continuity gaps
Then a second column: what happened. Repaired in place, redrafted, held for you, or returned as a note.
And a third column, the one that matters most: anything that reached a finished chapter with no flag at all. That column is the honest measure of a consistency system, and the reason to read your chapters rather than take any tool's word for it. The checks cross-reference each drafted chapter against the facts in the Story Bible, and the facts you have confirmed are the ones a chapter is held to.
How Logic-Locking Handles It
Four parts of the system do the work:
1. Persistent Fact Database
Character details, relationship changes, faction decisions and world rules are stored as structured facts, proposed from each chapter after it is drafted, for you to confirm or edit. This is not a summary or a bullet list. It is a store the checks can query, and each fact carries the chapter it came from.
Get writing tips that actually help.
Weekly insights on novel consistency, world-building, and writing a novel with a language model. No spam.
2. Relevance Scoring
Hundreds of facts will not fit into every chapter prompt. The system picks the facts most relevant to the chapter being drafted. Drafting a romance scene? Relationship history and personality traits come first, and faction politics only where they matter.
3. Post-Generation Verification
After each chapter is drafted, the checks compare it with the Story Bible. A contradiction of a fact you wrote or confirmed is repaired in place or the chapter is redrafted, up to 2 times, and the chapter is held for you if it is still there. A judgment call, or a fact the system inferred that you never confirmed, comes back as a note that quotes the line.
4. Locked Core Facts
Some facts you lock: eye color, a death, a magic rule, the geography of the world. What the system extracts does not overwrite a fact you locked. Other facts are meant to change: personality, relationship status, faction alliances. The Story Bible keeps the current state and the chapter each change came from.
What This Means for Writers
If you want to write a 50,000+ word AI novel, plan for the fact that a chat window is not built for it.
A general chat assistant, or a writing assistant with no store of facts, tends to drift as a book grows past what it is shown at once. Somewhere past chapter 20, you can end up spending more time fixing continuity than you would have spent writing.
A better model alone does not fix this. The fix is architecture: a store of facts outside the chat, and a check against it after every draft.
Our 25-chapter walkthrough shows the same method at half the length. Run either one on your own book and keep the log.
A 50-chapter book with complex characters, subplots, faction politics, a magic system and a timeline is a lot to hold in your head. The Story Bible holds it with you, so you are not tracking every detail in spreadsheets, and the checks tell you where to look.
Ready to Test It Yourself?
You do not need to take our word for it, and you should not. Start a project on Novarrium and push it past chapter 20. Watch the Story Bible grow, confirm the facts that matter, and read what the checks flag after each chapter is drafted.
Your first three chapters are free. No credit card required. Start writing your novel and keep your own failure log.
122 writers. 1.3M words tracked.
Bring your draft and Novarrium proposes a Story Bible from it: the characters, places and rules it finds, each with the line it came from, for you to review. When a new chapter breaks a fact you locked, it is repaired or held for you.
Importing your draft is free, and so are your next 3 drafted chapters, each held to your Story Bible. After that, drafting is on a paid plan.