Part 2 of the Can AI Write a Book? benchmark series. Six runs: qwen3:8b and llama3.1:8b, Q4_K_M, 32,768 context, reasoning off, 30 chapters × 3,000 words, identical control prompt, BookyAI v1.9.11–v1.9.12, Ollama 0.33.3.
Read this part first. Every number below is a single run on a single machine. One run each. Local generation is stochastic — same prompt, same model, same settings, different book every time. To say anything solid you’d want roughly ten runs per configuration, and I have one. Treat this as a starting point and a set of methods, not a verdict.
If you run these tests yourself, send me your numbers. The scripts, the control prompt, and the raw CSV matrix are all on GitHub. I will publish what you send, including results that contradict mine. Especially those.
What I said was going to happen
In the pillar post I laid out a hypothesis and named the test that would break it.
The RTX 4070 held qwen3:8b entirely on the graphics card and produced a clean run. The RTX 3070 spilled 41% of the work onto the CPU and produced four collapsed chapters. So I proposed that partial CPU offload doesn’t just cost you speed, it costs you coherence — and I said the RTX 2060, spilling 73% onto the CPU, would be the decisive test. If it came back worse again, the pattern was real.
It came back better. Better than the 3070, and better than the 4070 that never left the GPU at all.
Then I ran a second model on the same three machines and got the opposite answer.
Model one: qwen3:8b
| boo-4070 | trex-3070 | backoffice-2060 | |
|---|---|---|---|
| GPU (VRAM) | RTX 4070 (12GB) | RTX 3070 (8GB) | RTX 2060 (6GB) |
| Work pushed to CPU | 0% | 41% | 73% |
| Footprint reported by Ollama | 9.8 GB | 10 GB | 15 GB |
| Wall clock, ch1 → ch30 | 26.9 min | 460.5 min (7.7 h) | 975.0 min (16.3 h) |
| Median chapter | 53.5 s | 349.3 s | 1,742.3 s (29 min) |
| Generation speed | 68.3 tok/s | 6.03 tok/s | 2.22 tok/s |
| Body word count (ch1–30) | 89,758 | 128,677 | 100,719 |
| Target adherence | 100% | 143% | 112% |
| Chapters within ±20% of 3,000 words | 11 / 30 | 9 / 30 | 20 / 30 |
| Longest chapter | 6,764 | 27,970 | 6,329 |
| Internal repetition (body) | 45.5% | 68.5% | 40.2% |
| Chapters above 90% self-repetition | 0 | 4 | 0 |
| Chapters flagged truncated | 0 | 1 (ch19) | 0 |
| Drift, ch30 vs ch1 | 0.3% | 0.1% | 0.0% |
The oldest card in the set, with the smallest VRAM, the largest memory footprint, and the most work pushed onto a four-core laptop CPU, produced the least repetitive draft. It also held chapter length better than either of the other two: twenty of its thirty chapters landed within 20% of target, against eleven on the 4070.
It took sixteen hours and sixteen minutes to do it, versus twenty-seven minutes on the 4070. A 36× difference in time and a 5-point difference in repetition, in the 2060’s favour.
Rank the machines by CPU offload — 0%, 41%, 73% — and then by repetition — 45.5%, 68.5%, 40.2%. The machine in the middle of the offload range is the one that fell apart, and both ends came out fine. There’s no monotonic relationship available here, however I squint at it. The hypothesis died.
Model two: llama3.1:8b
Same prompt. Same context size. Same three machines. Same week.
| boo-4070 | trex-3070 | backoffice-2060 | |
|---|---|---|---|
| Work pushed to CPU | 0% | 34% | 71% |
| Footprint reported by Ollama | 9.1 GB | 9.5 GB | 14 GB |
| Wall clock, ch1 → ch30 | 55.7 min | 661.6 min (11.0 h) | 3,898.1 min (65.0 h) |
| Median chapter | 40.3 s | 363.4 s | 991.6 s (16.5 min) |
| Slowest single chapter | 785 s | 6,015 s | 66,520 s (18.5 h) |
| Generation speed | 34.2 tok/s | 4.45 tok/s | 0.80 tok/s |
| Body word count (ch1–30) | 86,507 | 133,447 | 141,636 |
| Target adherence | 96% | 148% | 157% |
| Chapters within ±20% of 3,000 words | 0 / 30 | 1 / 30 | 0 / 30 |
| Median chapter length | 814 w | 938 w | 898 w |
| Longest chapter | 27,729 | 28,774 | 28,367 |
| Internal repetition (body) | 75.2% | 86.0% | 87.2% |
| Chapters above 90% self-repetition | 3 | 4 | 6 |
| Chapters flagged truncated | 4 | 2 | 6 |
Now rank by offload — 0%, 34%, 71% — and by repetition — 75.2%, 86.0%, 87.2%. That’s monotonic, in exactly the direction I originally predicted. More offload, more repetition, more collapsed chapters.
So I have one model where the hypothesis is impossible and one where it holds cleanly. With three data points, a random ordering comes out monotonic in the predicted direction one time in six. I cannot distinguish “the offload effect is real and model-dependent” from “I flipped a coin twice” without repeat runs, and I don’t have them.
That’s an unsatisfying place to leave it and it’s where the evidence actually leaves it.
What does survive
Put the two tables next to each other and the largest number in the whole experiment isn’t a hardware effect.
| Repetition | qwen3:8b | llama3.1:8b | swing from changing model |
|---|---|---|---|
| boo-4070 | 45.5% | 75.2% | 29.7 points |
| trex-3070 | 68.5% | 86.0% | 17.5 points |
| backoffice-2060 | 40.2% | 87.2% | 47.0 points |
Changing the model on the 2060 moved repetition by 47 points. The entire spread across all three machines running qwen3:8b is 28 points, and across all three running llama3.1:8b it’s 13.
Model choice sets the floor. Hardware moves you around within it. That’s the finding I’d defend, and the practical advice follows from it:
Uncheck the reasoning box. Then spend your effort choosing the model, not the graphics card.
Fitting the model entirely in VRAM took generation time from sixteen hours to twenty-seven minutes, which is an enormous practical difference if you’re iterating. It did not reliably make the prose better.
(Reasoning is off in every run from here forward and I’m done writing about the checkbox. On the 4070 it was a 12.8× speedup — 344 minutes down to 27 — and it took repetition from 90.9% to 45.5%. It’s part of the control setup now.)
The two models fail in completely different ways
This is the part the aggregate numbers hide.
qwen3:8b on the 3070 had four seizures. Not a book that degraded — a book that fell into a loop, climbed out, fell in again, and climbed out for good.
| Chapter | Words (target 3,000) | Self-repetition |
|---|---|---|
| 16 | 7,907 | 93.7% |
| 17 | 13,854 | 99.4% |
| 18 | 2,380 | 71.9% |
| 19 | 27,970 | 99.6% |
| 20 | 16,446 | 98.8% |
Chapter 18 sits between two of the blowouts at a perfectly ordinary 2,380 words. Chapters 21 through 27 come back at around 2,000 words each, one of them at 0.7% self-repetition — the cleanest chapter in the entire run, generated immediately after the worst one. Whatever causes that, it isn’t cumulative context pressure.
Those four chapters account for 66,182 words — 51% of the body and 58% of the total generation time. Take them out and the remaining 26 chapters total 62,495 words, or 80% of target. The 143% adherence figure isn’t a book that ran long; it’s a book that ran short with four blowouts glued on.
llama3.1:8b did something else entirely, on all three machines. It never wrote a chapter of the requested length. Median chapter: 814, 938 and 898 words against a 3,000-word target. Across ninety chapters on three machines, exactly one landed within 20% of target, and it was chapter one. Then three to six chapters per run blew out to 27,000-plus words at 96–99% self-repetition, carrying 74% to 83% of each book’s total word count.
Chronic undershoot punctuated by catastrophic blowouts. That signature was identical on a card that held the whole model in VRAM and on one that pushed 71% to the CPU, which is the single strongest reason to think the failure belongs to the model.
The accident that turned into a control
Partway through the 3070’s llama3.1:8b run, BookyAI fell back to its configured backup model and generated chapter 6 with Gemini 3.1 Flash Lite instead of the local model.
That chapter came out at 3,070 words with 0.0% self-repetition — the only chapter in the entire run to hit target length. I regenerated it locally to keep the run clean, and llama3.1:8b produced 849 words for the same prompt in the same position in the same book.
One chapter, a different model class, not a controlled comparison. But it’s the cleanest illustration I have of what the local 8B model was failing to do, and it landed by accident in the middle of a run I was already measuring.
Every figure in the 3070 column above is from the all-local version, with the Gemini chapter replaced. Swapping it out also produced a methodological surprise: removing 2,221 clean words raised the book’s repetition rate, without changing a single repeated passage. Repetition percentages are sensitive to the denominator. Report word count next to every one of them.
One wrinkle I can’t fully clean up: the 3070’s wall-clock figure includes the roughly 90 minutes BookyAI spent failing over and generating that chapter with Gemini. The timing is from the original run; the text is from the local regeneration. They describe very slightly different files and I’d rather say so than quietly pick one.
What the slow machine actually costs you
Time, and a specific kind of time.
On the 4070 you get a qwen3:8b chapter every 53 seconds and a finished manuscript over a lunch break. On the 2060 the median chapter is 29 minutes, and with llama3.1:8b the whole run took 65 hours — two and a half days — including one chapter that alone took 18 and a half hours.
You don’t watch that. You start it and check back tomorrow. It means roughly one experiment per day instead of ten per hour, and iteration speed is most of what makes this kind of work tractable.
But it’s patience, not money, and patience is the resource this series is about.
On money: I still owe you real numbers and I want to be careful about whose numbers they are. The meter is arriving, and what I’ll publish is kWh per finished book, because that’s a property of the machine rather than of my billing arrangement. The trex-3070 lives in a coworking office where electricity is included in the rent, so the marginal cost of that 65-hour run was zero to me — which tells you nothing useful, and would be actively misleading if I reported it as a cost. Once I have kWh I’ll show what each run would cost at the 13–18¢/kWh West Lafayette residential range, so you can substitute your own rate. The $0.96 figure in the pillar post was an estimate from the misconfigured reasoning-on run and should be read as an illustration, not a floor.
What none of them did
Write a usable book.
The best repetition number across six runs is 40.2%. Two out of every five ten-word windows in that manuscript appear somewhere else in the same manuscript. My own edited novel, after 44 hours of work, measures 0.2%.
Top repeated passage from each qwen3:8b run, by the number of overlapping ten-word windows it occupies:
- 2060: “The AI had taken on a life of its own” — 87 windows
- 4070: “And as she sat there, staring at the screen, she” — 43 windows
- 3070: “was finally beginning to find her way back to herself” — 221 windows
The 2060’s single worst phrase is twice as entrenched as the 4070’s even though its overall repetition is lower. Winning on the aggregate doesn’t mean winning on every measure, and I’d want more runs before telling anyone the 2060 is “better” at anything.
The honest summary of all six runs is the same summary: a local 8B model at 32k context will hand you a manuscript-shaped object of roughly the right length, and between 40% and 87% of it is recycled. The editing tax is the whole story, and that’s the next post.
Bugs and method corrections
Publishing these is part of the method, so:
Whole-file measurements flatter the model. bookdiff repetition run across an entire export includes the back matter, which is less repetitive than the novel and drags the number down. Measured both ways, the qwen3:8b books read:
| whole file | body only (--chapters-only) | |
|---|---|---|
| 2060 | 38.8% | 40.2% |
| 4070 | 42.5% | 45.5% |
| 3070 | 65.4% | 68.5% |
A 1.4 to 3.1 point difference, always in the same direction. Every repetition figure in this series is now the --chapters-only number.
--chapters-only was only half-fixed, and I caught it late. Per-chapter endnote entries carry their own Chapter N headings, so the splitter detected them as chapters and they survived a trim applied inside chapters. Reports came back claiming 60 chapters in a 30-chapter book. The fix is to truncate the raw text at the first back-matter heading before splitting.
Three of the six figures in this post were originally measured with the broken version. I re-measured all three with the current script and every one moved up, which is what the back-matter effect predicts:
| published earlier | re-measured | |
|---|---|---|
4070, qwen3:8b | 45.4% | 45.5% |
4070, llama3.1:8b | 74.6% | 75.2% |
3070, llama3.1:8b | 83.7% | 86.0% |
Small moves, same direction, no conclusion changed. The tables above use the re-measured numbers. If you took figures from an earlier post in this series, these are the ones to use.
I published the 3070’s speed with an inconsistent formula. The pillar post says 6.2 tok/s. That divided all chapter tokens, chapter 1 included, by a clock that starts when chapter 1 finished. chaptertimes.ps1 can’t see when chapter 1 started, so chapter 1’s tokens have to come out of the numerator too. The consistent figure is 6.03 tok/s, and the 4070’s 68.3 was already calculated that way.
Tokens per second is depressed by short chapters. The 4070’s llama3.1:8b figure of 34.2 tok/s looks like the model running at half the speed of qwen3:8b on the same card. It isn’t. Per-chapter timing includes fixed overhead, which dominates an 800-word chapter — during the blown-out chapters, where generation actually runs long, the same model hit 57.9 to 70.0 tok/s. Don’t compare tok/s across runs with very different chapter lengths.
The pillar says 0 of 42 chapters were flagged truncated on the 3070. Chapter 19 was flagged, and given that it’s 27,970 words of a repeating paragraph, the flag was correct. Corrected to 1.
The emoji fix changed an old number. qwen3:8b likes to end chapters with 🌟 or 🌿, and the truncation check was reading a trailing emoji as a missing full stop. The script now strips decorative characters before testing for terminal punctuation, which reclassified two 4070 chapters. That run’s truncation count drops from 2 to 0.
Two runs are unusable and I’m keeping them anyway. The first thinking-mode attempts on the 2060 and the 3070 both had BookyAI upgraded mid-run (v1.9.10 → v1.9.11 → v1.9.12) and Ollama updated underneath them. Both aborted — one stalled on a chapter that BookyAI kept flagging as a repeat of the previous one and tried to rewrite indefinitely. They’re in the matrix as failure-mode documentation with ABORTED status and they contribute nothing to any comparison. Version drift mid-run invalidates a benchmark run completely, and it’s easier to do by accident than you’d think.
What this means if you don’t have a good graphics card
This is the part I actually care about, and it’s why the model-versus-machine result matters more than it sounds.
If coherence is mostly a property of the model, then the expensive card buys you iteration speed, not a better book. And iteration speed is a researcher’s problem, not a writer’s problem. Someone who wants one novel draft, not forty, doesn’t need to iterate. They need to start a run and come back tomorrow.
The backoffice-2060 is a card from early 2019 in a machine with a four-core laptop CPU. It produced the least repetitive qwen3:8b draft in this experiment and held chapter length better than either faster machine. It took sixteen hours instead of twenty-seven minutes. If you already own a machine like that, the thing standing between you and a 90,000-word draft is patience and a power bill, not a purchase.
Two honest caveats on that, because I don’t want to oversell it.
The machine being “on anyway” is not the same as the run being free. Idle draw and sustained-inference draw are very different numbers, and a 65-hour run at load is a real cost even on a computer that was already plugged in. That’s precisely what the meter is for, and I’m not going to claim a number before I have one.
And I’ve been running a browser with a dozen tabs open on all three machines throughout — GitHub in one, the Ollama model listing in another — which is already in my confounds list as an uncontrolled variable. Whether you can comfortably answer email on the 2060 while it generates is a real usability question and I’ve been accidentally testing it for two weeks. It deserves a proper measurement rather than an anecdote, so I’ll do that one on purpose.
Where the local path runs out
There’s a ceiling on all of that, and it’s worth being precise about where it sits, because it isn’t where most people assume.
It isn’t VRAM. The 2060 has 6 GB of it and is currently generating a novel with a 30B-class model that reports an 18 GB footprint. Ollama spills whatever doesn’t fit onto the CPU and system RAM, and the result is slow, not impossible. Every “your card isn’t big enough” post you’ve read is describing a speed problem.
The real wall is system RAM, and it’s absolute. llama4:16x17b is a 67 GB model. Two of my three machines have 64 GB of RAM. On those two it does not load slowly — it does not load. Ollama returns a timeout, ollama ps shows nothing, because the model never becomes resident anywhere. There’s nothing to troubleshoot. It’s arithmetic.
So the two ceilings do different jobs. VRAM decides how fast. System RAM decides whether. And the second one is cheap to raise — another 64 GB of DDR5 costs less than a mid-range graphics card, and no consumer card on the market could hold that model in VRAM at any price.
Here’s the uncomfortable part. This series’ central result so far is that the model matters more than the machine. The best book I’ve produced came from the largest model I’ve run. That’s one 30B model beating two 8B models — a hypothesis, not a finding, and exactly the kind of thing I’d want repeat runs before believing. But if it holds, and if the trend continues upward, then the local path ends at a specific and knowable point: the largest model that fits in the most RAM you can put in a box you own.
Past that point, more patience doesn’t help. No amount of waiting loads a 67 GB model into 64 GB.
The other door
Which is where the second half of this project comes in, and it inverts the hardware question entirely.
If the model is what matters, you don’t actually need to run the model. You need to reach it. Services like OpenRouter sell access to models far larger than anything I can load locally, priced per token, and the only thing your computer has to do is send an HTTP request and hold the result. A netbook can do that. A ten-year-old laptop can do that. The machine requirement collapses to “can open a browser.”
That trades one cost for another — you’re renting compute instead of owning it, you’re sending your manuscript to someone else’s server, and you’re paying per book instead of per kilowatt-hour. All three of those are real and two of them are why I started with local models in the first place.
But the comparison is the whole question for anyone this series is written for, and I’ve never seen it done honestly at novel length. So that’s the next track: I’m going to put real money into OpenRouter credit, run the identical control prompt through models too large for any machine I own, and measure the output with the same scripts. Then set it against the local runs.
The question I actually want answered: at what point does twenty dollars of API credit beat sixty-five hours of your own electricity on a computer you already have? I don’t know yet. It may turn out that the honest advice for someone locked out of a $20/month subscription is a $3 API bill and a laptop, not a graphics card. It may turn out the local draft is good enough and free enough that it wins. Either answer is useful, and neither is one I can guess.
Where the series goes next
The hardware sweep has told me what it’s going to tell me. Three machines, two models, and the largest effect in the data was the model. So I’m turning the experiment around: less hardware, more model and settings.
Two things change.
First, quality becomes the primary measurement, not repetition. Repetition was always a proxy — easy to compute, and it caught the catastrophic failures, which is what the first six runs mostly were. But a reader doesn’t want a low repetition score, they want a draft they can edit without rewriting, or publish as generated. The 30B-class model I’m running now is clean enough that repetition has stopped discriminating, which means it’s time for a better instrument. That’s the next post.
Second, a context-window experiment. Every run so far has used num_ctx 32768 because that’s what I set in the control prompt on day one, and I’ve never tested whether it’s the right number. A smaller window means less to attend to, a smaller KV cache, and a smaller memory footprint — which might help coherence, hurt it, or do nothing. Nobody publishes this for local models at novel length.
| Run | Machine | Model | num_ctx |
|---|---|---|---|
| D1 | boo-4070 | muse-glimmer:30b | 16,384 |
| D2 | trex-3070 | muse-glimmer:30b | 16,384 |
| D3 | backoffice-2060 | muse-glimmer:30b | 16,384 |
Each pairs directly against the 32,768 run of the same model on the same machine, so the only thing that changes is the context window. I’m starting with muse-glimmer:30b because it’s produced the best output in the series so far by a wide margin.
That set does double duty. It’s also the falsification test I promised in the pillar post: I’ll record the ollama ps footprint at each context size, and if the memory footprint falls roughly in proportion when the window halves, uncompressed KV cache explains the 2060’s mysterious 15 GB. If it doesn’t fall, it’s offload accounting and I’ve been reading that number wrong.
Still outstanding
- Repeat runs. The big one. With n=1 per cell I can’t separate a real offload gradient in
llama3.1:8bfrom a coin landing the same way twice. - The OS confound. trex-3070 runs Windows 10 and can’t upgrade. It’s also the machine
qwen3:8bbroke on. I have no evidence that’s related and no way to test it without a fourth machine. The antivirus setup also differs across the three and I’m not changing it mid-series. - Actual wattage measurement instead of estimates.
- The OpenRouter comparison, priced and measured against these local runs with the same scripts. That’s a separate post and it needs a budget, not a machine.
- Whether bigger really is better. The 30B result is one model. Until I’ve run more of them, “the larger model wrote the better book” is a pattern with n=1 behind it, and the whole local-versus-API argument above rests on it.
Disclaimer
A quick note on how I wrote this: I used Claude Opus 5 to help me work through the results of these experiments, check my reasoning, and shape some of the analysis in this post. I’m learning as I go, and that is honestly one of my favorite parts of this project. I’m stretching my knowledge of local AI, benchmarking, hardware, and data analysis quite a bit beyond where I started, and I’m excited to keep learning, testing my assumptions, finding mistakes, and getting better at it as I go. These experiments are exploratory, not definitive research, and I’ll keep updating my conclusions as I collect more data.
If you’ve run a local model past 50,000 words, I want your repetition number. Run bookdiff.py repetition yourbook.md --chapters-only, tell me the model, the context size, the word count, and your GPU/CPU split from ollama ps, and send it over. Six runs on three machines is an anecdote. Sixty runs from thirty people is a benchmark.
0 Comments