Last updated September 12, 2026. This is a live experiment; numbers change as runs complete. For the most recent results and my journey using AI to write a book using local AI, check out posts in my Local AI Benchmarks category
On August 19, I generated a full-length book with AI, uploaded it to Amazon, and someone promptly left a one-star review calling it “AI slop.” They were completely right. (You can see that original project documented in my first write-up over at Can AI Write a Novel in a Day? My BookyAI Experiment with The Affirmation Glitch, and if you want to see the original artifact that earned that review, it’s still live on Amazon).
That original experiment was a chaotic, fascinating baseline. But it left me with a burning question: was the output terrible because local models are fundamentally incapable of longform narrative, or because my setup was flawed?
This post is the first installment in an ongoing benchmarking series designed to answer that question with hard numbers. Because the experiment is actively running across three separate computers as I write this, today’s post focuses on the fresh data from my primary machine (the RTX 4070), while the slower nodes continue chugging away overnight.
We need to talk about this because the digital divide we all ignored in the 90s is happening all over again, just wearing a much more expensive trench coat.
The Digital Divide is Now the AI Divide
Back in September 2024, I gave a talk about AI and ended up recording a voice note afterward that I haven’t been able to stop thinking about. When I worked at Greenleaf Market on the north side, I saw firsthand that for a lot of people, high-speed home internet simply isn’t a given. Their only window to the digital world is an older smartphone—like an iPhone 8—or maybe pooling Wi-Fi with neighbors, sitting outside a coffee shop, or taking the bus to the public library.
If you are already struggling to afford a basic internet connection, how are you supposed to pay a $20 monthly subscription for ChatGPT Plus? You aren’t. At work, people without kids or with dual incomes (like me) can easily toss $20 a month at an AI tool to get ahead, buy experimental lifetime software deals, and learn the ropes of the future economy. But if you don’t even have reliable Wi-Fi, you aren’t even making it to first base. AI requires the internet, meaning the exact same under-resourced communities are getting locked out of the next massive technological shift before they even know it’s happening.
I’m doing this massive, deeply nerdy experiment because everyone benchmarking local AI models is optimizing for speed on $3,000 graphics cards. I want to know what happens if your software budget is zero dollars and you just have patience and an old computer.
The Five Big Questions
- How hard is it, really, to write a book with AI—and what actually comes out of it?
- Is AI genuinely “there” yet for longform narrative continuity?
- What kind of editing tax does the output need, and how many human hours does it take to make it readable?
- Can this be done for free? What is the actual minimum hardware required?
- What does this mean for people completely locked out of the $20/month AI subscription world?
Before we dive into the numbers, I want to be radically transparent: this entire experiment is pushing the absolute edges of my technical knowledge. The Python scripts, the PowerShell code, and the testing framework were all designed and written with heavy assistance from Claude AI, and I’m using Gemini right now to help me draft and structure this post. I am learning alot as I go, and it is a massive privilege to have access to these models to teach me how their own backends work.
The Control Setup: Standardizing The Affirmation Glitch
To make this benchmark clean and reproducible, I took the exact narrative premise from my original August 19 run and locked it down using standard dropdown options inside BookyAI.
BookyAI has been a fantastic platform for this experiment. I originally bought it because they offer a lifetime deal for under $100, freeing users from suffocating monthly “credit systems.” It lets you hook directly into your local Ollama instance (I’m running version 0.33.3), OpenRouter, or direct API keys for Claude, Gemini, Grok, and OpenAI. Their team is very responsive—when I reported an incomplete-generation bug a few weeks ago, they squashed it within 24 hours.
For this benchmark, quantization was verified as Q4_K_M using ollama show. Temperature and top_p were held constant at BookyAI’s default generation values. Back-matter options (appendices and author notes) were kept turned ON for all runs to ensure direct comparability.
I fed the generator the same prompt as my first set of experiments. I changed the voice, tone and genre to fit with BookyAI’s dropdowns:
- Genre & Subgenre: Fantasy / Cozy Fantasy
- Target Audience: Women
- Tone: Inspirational and uplifting
- Writing Style: Vivid and immersive
- Narration: Third person
- Target Scope: 30 chapters × 3,000 words each (Target: 90,000 words total)
I locked the context window (in BookyAI and also oLlama) to 32K tokens (which I think is actually 32,768 tokens) across all machines. Why? Because context memory scales linearly, and Ollama allocates it up front whether you use it or not, which crowds the actual model layers right off the GPU. In my V1 experiment, I ran at 256k context. This means the chapter truncation I diagnosed as a “socket timeout” bug last month actually had two potential causes running at once. Fixing the context size isolates the variables cleanly.
Here is the hardware matrix across the three test rigs:
| Machine | GPU (VRAM) | System RAM | CPU | Cores / Threads | Memory Channels | OS Version, Build, Notes |
|---|---|---|---|---|---|---|
| boo-4070 | RTX 4070 (12GB) | 128GB | i9-14900KF | 24c / 32t | Dual-channel | Windows 11 Pro 25H2 26200.9445 |
| trex-3070 | RTX 3070 (8GB) | 64GB | i9-10980XE | 18c / 36t | Quad-channel | Windows 10 Home 22H2 19045.6466 Cannot upgrade to Win 11 |
| backoffice-2060 | RTX 2060 (6GB) | 64GB | i7-1165G7 | 4c / 8t | Dual-channel | Windows 11 Pro 25H2 26200.9445 |
Notice that trex-3070 has quad-channel memory and 18 CPU cores. When a model spills off the GPU into system RAM, memory bandwidth becomes the main bottleneck—meaning this older, cheaper machine might actually beat the RTX 4070 on larger models.
One variable I nearly missed: the three machines aren’t running the same operating system. Two are on Windows 11 Pro 25H2. trex-3070 is on Windows 10 22H2 and cannot be upgraded — its Xeon-class i9-10980XE isn’t on Microsoft’s supported CPU list, despite being an 18-core processor that outclasses the chips in both other machines.
That’s a confound I have to disclose: Windows 10 and 11 differ in GPU scheduling and memory management, so any 3070-versus-4070 difference is hardware plus OS, not hardware alone.
It’s also the thesis of this series in miniature. Windows 10 stopped receiving security updates in October 2025. The machine most likely to end up in the hands of someone who can’t afford a subscription is the one that’s been declared obsolete by a compatibility list rather than by its actual capability — and it still has 18 cores and quad-channel memory.
The Hidden Hardware Penalty: 15GB on a 6GB Card
Here is the first major discovery: an older graphics card doesn’t just run slower—it actively demands drastically more system memory to do the exact same job.
I loaded a standard 5.2GB model (qwen3:8b) onto all three test rigs with 32k context. On the newer RTX 4070, the total process footprint reported by Ollama was 9.8 GB (100% on GPU). But on the older RTX 2060? The reported memory footprint ballooned to 15 GB (73% CPU / 27% GPU).
There are two competing explanations for why a 5.2GB model demands 15GB on an older card:
- Uncompressed KV Cache: Newer GPU architectures (Ampere and Ada) support hardware-level attention optimizations (“flash attention”) and quantized KV caching. The older Turing architecture on the 2060 lacks these features, storing the entire 32k context uncompressed.
- CPU Layer Offload Accounting: When Ollama splits a model across GPU and CPU, the reported process size may include duplicated layer buffers in system RAM.
I’m going to run a simple test on the 2060 to falsify this: dropping num_ctx down to 8k and 16k. If the reported size falls in direct proportion to the context length, the uncompressed KV cache is the culprit.
The “Thinking” Glitch: Reading the Tooltip
While setting up the baseline runs on boo-4070, I made a classic user error. Certain instruction-tuned models output internal “reasoning” traces before generating story text. BookyAI literally has a tooltip next to the reasoning checkbox warning that prose doesn’t need thinking traces and that leaving it enabled can cause chapters to take hours on local hardware.
I ignored the tooltip. The model spent hours generating thousands of hidden reasoning tokens. In Chapter 16, the reasoning channel literally leaked into the narrative prose, leaving a bare </think> tag right in the middle of a scene.
Here is what happened when I ran the exact same model with reasoning toggled ON versus OFF on boo-4070:
| Metric | Thinking ON (Run A3) | Thinking OFF (Run A3b) | Impact / Delta |
| Wall-Clock Generation Time | 344.2 minutes (~5.7 hours) | 26.9 minutes | 12.8× faster execution |
| Total Word Count (includes appendices) | 333,227 words | 89,758 words | Dead-on 90k target adherence |
| Target Length Accuracy | 370% (Wild overshoot) | 100% (Exact target) | Perfect length control |
| Generation Speed | 21.5 tokens/sec | 68.3 tokens/sec | 3.2× faster throughput |
| Internal Repetition Rate (Body) | 90.9% | 45.4% | Repetition cut in half |
On the 4070 machine, unchecking that single documented setting dropped generation time from nearly six hours down to 26.9 minutes, and brought the body word count directly to 89,758 words—hitting our 90,000-word target with 100% accuracy. (See the update below — it did not work the same way on my 3070 machine.)
The Actual Cost: $0.02 vs. $20/Month
Let’s answer the core question about the AI divide: what did that 26.9-minute run actually cost?
On boo-4070, the system pulled roughly 250 watts during generation. At 0.45 hours of runtime, that equals roughly 0.11 kWh of electricity. At standard Indiana residential power rates (~$0.15 per kWh), generating a complete 89,758-word novel cost less than two cents ($0.02). The 6 year old 2060 is on chapter 14 out of 30, almost 24 hours later (with reasoning turned on) and currently at approximately $0.96 of electricity. Worth flagging: that 2060 run has reasoning on — the same setting that cost the 4070 six hours. So $0.96 is the cost of the misconfigured run, not the floor. The corrected 2060 number is still generating, and I’ll update this line when it lands.
That’s the thesis in a single number: two cents of electricity versus $20 a month. Even the slow, misconfigured run on six-year-old hardware came in under a dollar.
However, electricity isn’t the whole floor. A used RTX 2060 graphics card runs about $150 — roughly eight months of ChatGPT Plus before you break even, and you need a desktop to put it in. A used computer with the 2060 graphics card in it costs about $400-700 (more than the cost of ChatGPT for a year). My claim isn’t “anyone can do this.” It’s that the floor is a one-time hardware cost instead of a recurring subscription, which is a very different barrier for someone whose income is low. Also, when one is not using the 2060 for generating longform content or running OpenWebUI (as a free alternative to ChatGPT, Claude, Gemini etc.), the computer can be used for work, job-hunting, learning and everything else.
Update, September 12: My Second Machine Contradicted Me
I published the section above after testing one machine. Then the RTX 3070 finished the same run and told a different story.
Same model. Same prompt. Same 32k context. Reasoning off on both.
| boo-4070 (A3b) | trex-3070 (A2b) | |
|---|---|---|
| Reasoning | OFF | OFF |
| Wall clock (ch1 → ch30) | 26.9 min | 460.5 min (7.7 hours) |
| Body word count | 89,758 | 128,836 |
| Target adherence | 100% | 143% |
| Generation speed | 68.3 tok/sec | 6.2 tok/sec |
| Internal repetition | 45.4% | 68.5% |
| Chapters flagged truncated | 0 of 42 | 0 of 42 |
| GPU / CPU split | 100% GPU | 41% / 59% CPU |
Unchecking the box did not save the 3070. Four chapters collapsed entirely:
| Chapter | Words (target 3,000) | Self-repetition |
|---|---|---|
| 16 | 7,907 | 93.7% |
| 17 | 13,854 | 99.4% |
| 19 | 27,965 | 99.6% |
| 20 | 16,446 | 98.8% |
Chapter 19 is 27,965 words against a 3,000-word target, and 99.6% of its 10-word windows appear more than once. It isn’t a long chapter. It’s one paragraph repeated for nine times the intended length.
Those four chapters total 66,172 words — 51% of the entire book. Remove them and the remaining 26 chapters come to 62,664 words, which is 80% of target. So this wasn’t a manuscript that ran long. It was 26 reasonable chapters with four blowouts glued on.
What I got wrong, and what I now think
I wrote that unchecking the reasoning box fixed the runaway. More precisely: it was necessary but not sufficient. It fixed the machine where the model sat entirely on the GPU. It did not fix the machine that had to spill 59% of the work onto the CPU.
Which points at a better hypothesis than the checkbox. The difference between these two runs isn’t a setting — it’s how much of the model fit on the graphics card.
| Machine | GPU / CPU split | Reasoning OFF result |
|---|---|---|
| boo-4070 | 100% GPU | clean, 45.4% repetition |
| trex-3070 | 41% / 59% CPU | 4 chapters degenerate, 68.5% |
| backoffice-2060 | 27% / 73% CPU | still running |
If the 2060 is worse again, that’s a real finding and not a small one: partial CPU offload doesn’t just make generation slower, it makes the output worse. Everyone knows offloading costs speed. Nobody talks about it costing coherence.
I can’t prove that yet, and there’s a competing explanation I can’t rule out: generation is stochastic, and the 4070 may simply have gotten a lucky run. Only repeating both runs distinguishes the two. That test is on the list.
The 2060 is finishing now. When all three machines have completed the identical configuration, I’ll publish the three-way comparison as its own post — and if the offload hypothesis holds, it changes the practical advice in this series from “uncheck the box” to “make sure your model fits in VRAM, and if it doesn’t, expect the prose to suffer and not just the clock.”
Quantifying AI Slop: Repetition and Speed Decay
While turning off reasoning fixed the runaway word count, the prose quality remained a major issue. Standard editing software like AutoCrit won’t notice if an unedited AI writes “And as she sat there, staring at the screen, she” forty-three times across thirty chapters.
To measure this, Claude and I wrote bookdiff.py, a Python script that analyzes overlapping 10-word text windows across a manuscript to calculate an Internal Repetition Rate.
(Note on method: bookdiff counts overlapping 10-word positions. A single 20-word repeating phrase contains 11 overlapping windows, so a literal text search in Word or VS Code will return roughly one-tenth as many hits as the window count).
Here is where the data stands:
- My own edited novel: 0.2%. It started as a Fable draft — my carryover script says 64.9% of its sentences are still word-for-word the AI’s — and 44 hours of editing brought repetition from AI-typical down to 0.2%. So this isn’t “human writing beats AI writing.” It’s the same prose after editing. That’s one data point (n=1); run bookdiff.py on a book you trust and tell me what you get.
qwen3:8bUnedited Output (Thinking OFF): 45.4% repetition rate. Nearly half of the generated novel consisted of recycled filler phrases.
As for generation speed, raw averages suggested a 14% slowdown over the course of the book (72.2 tok/s in Ch 2–11 down to 62.0 tok/s in Ch 21–30). But looking at the medians tells a subtle story: median throughput fell by 6.4% (71.5 tok/s down to 66.9 tok/s).
The drop was driven heavily by three outlier chapters (Ch 21, 29, and 30), which dropped to 38–53 tok/s. This happened because BookyAI’s internal repetition guard detected looping text and forced automatic chapter regenerations behind the scenes. A silent retry shows up in timestamp data as a slow chapter. So while KV-cache growth causes a mild ~6% throughput decay, automatic retry loops are what really bite into wall-clock time.
chaptertimes.ps1 was reading UTF-8 files as Windows-1252. PowerShell 5.1 doesn’t default to UTF-8, so every emoji and curly quote arrived as mojibake letters. qwen3:8b likes to end chapters with a decorative 🌟 or 🌿; the script saw a letter, concluded there was no terminal punctuation, and reported the chapter as cut off mid-sentence. Seventeen of my forty-two “truncated” chapters on the 3070 were nothing of the kind.
Worse, the bug was machine-dependent — PowerShell 7 on my Windows 11 boxes defaults to UTF-8, so the same script gave different answers on different computers in the same benchmark. Fixed by forcing the encoding explicitly.
Complete Transparency: Open-Source Code and Bugs
I believe in showing my work—including my mistakes. During early test runs, my scripts contained two embarrassing bugs:
bookdiff.pywas counting endnote headings as chapters and swallowing the back matter into Chapter 30, inflating the text scope.chaptertimes.ps1was counting back-matter files in the total chapter count and reported an 8,297-second idle gap (which was just me stepping away from the computer before generating appendices) as the “slowest chapter.”
I fixed both bugs live, re-ran the analysis, and verified the clean body word count.
I refuse to lock any of this research behind a squeeze page or lead-generation funnel. The public control prompt, analysis scripts, raw CSV matrix, and unedited generated text files are available on GitHub:
👉 GitHub Repository: github.com/HelloJessicaM/can-ai-write-a-book
You can download the tools, inspect the raw data, and run the math yourself. If you want quiet updates whenever a new machine or model batch finishes, you can join my low-frequency email list in the footer form—no pitches, no courses, no consulting funnels.
Upcoming Posts in This Series
- 🟢 (this post) — Can AI Write a Book?
- 🟢 Two Models, Three GPUs: The Model Mattered More Than the Machine
- 🟢 Is Bigger Better? What Happens When a Model Doesn’t Fit in VRAM?
- 🟡 Coming Soon: Dense vs. Mixture of Experts (MoE) on Budget Hardware
- 🟡 Coming Soon: How Much Editing Does AI Prose Actually Need? The Carryover Curve.
- 🟡 Coming Soon: Local GPUs vs. Cloud Free Tiers vs. Fable API: What $40 Actually Buys You
Have you ever tried pushing a local AI model to write something longer than a single scene, and at what point did you notice it start recycling its own prose?