What a Language Model Can and Cannot Do for Your Novel

I build machine learning systems for a living and I write in my spare time, which means I have spent an unusual amount of time on both sides of this question. Most of what is written about AI and fiction is either a sales pitch or a eulogy. I want to try for something duller and more useful: what these systems actually do, and therefore what they are and are not good for at a writing desk.

Start with the mechanism, because it explains everything else

A large language model is a very good next-token predictor. Given everything written so far, it produces a probability distribution over what comes next, and samples from it. That is genuinely all it is doing, and the fact that this produces coherent paragraphs at all remains one of the more surprising empirical results of the last decade.

But keep the mechanism in mind, because it predicts the failure modes exactly.

The model is drawn towards the centre of its training distribution. Asked for a description of grief, it will produce the average of an enormous quantity of writing about grief. That average is fluent, grammatical, appropriately paced, and completely unmemorable. It is the prose equivalent of a stock photograph. Good writing is almost by definition off-centre — the specific, slightly wrong detail that no one else would have chosen — and off-centre is what a distribution's mode is not.

It also has no stake in anything. Every sentence a good writer commits to closes off alternatives; it means the story is now this and not that. A sampler has no preference. Ask it for ten versions and it will happily give you ten, all plausible, none of them insisted upon. What reads as blandness in generated fiction is usually this: a total absence of commitment.

And it does not know what it does not know. There is no internal flag separating the parts it has effectively memorised from the parts it is improvising. Confidence is uniform. For fiction this matters less than for factual work, but it matters a great deal the moment your novel touches a real place, a real profession, or a real historical period.

Where the help is real

Given all that, the useful applications are the ones that do not require taste or commitment.

Generating options you will mostly reject. Twenty possible ways a scene could go wrong is a genuinely useful list even if nineteen are obvious, because the act of rejecting them sharpens what you actually want. This is what the machine's tendency towards the average is good for: it is an excellent map of the expected, which is a map of what to avoid.

Interrogating a draft you have already written. This is where I get the most value, and it is nearly the opposite of how these tools are usually marketed. Paste in a chapter and ask what a reader would be confused about, where the pacing sags, what this character seems to want, which details are being repeated. The answers are unreliable in detail and useful in aggregate. It is a reader with no social obligation to be kind and no memory of your intentions — which is what you actually need, because the problem with your own draft is always that you can still see what you meant.

Continuity and reference work. Which chapter did the sister first appear in, what colour did I say the car was, does the timeline in part two work. Tedious, mechanical, genuinely helped.

Unblocking by lowering the stakes of a first sentence. Sometimes a bad generated paragraph is enough to break the paralysis, because reacting against something is easier than starting from nothing. You then delete it. That is fine — it did its job.

Research direction, never research. It is good for finding out what you should be reading about, and unreliable as a source in itself. Treat every factual output as a lead requiring verification, because it will invent citations with perfect composure.

Where it actively hurts

Voice. Voice is the accumulation of a thousand small idiosyncratic choices, and the model's entire tendency is to smooth idiosyncrasy towards the mean. Writers who draft with heavy generation and then "edit for voice" usually end up with something that has the shape of their voice and none of the grain. Editing cannot put back what was never generated.

Structural decisions. Ask a model to fix a broken plot and it will produce a fix — conventional, tidy, and usually the wrong one, because it resolves the tension rather than finding what the tension is about. Plot problems are almost always character problems in disguise, and diagnosing that requires knowing what you are trying to say.

Surprise. This is the fundamental one. The thing you remember about a good book is the part you could not have predicted. A system optimised to predict text is structurally the wrong instrument for producing the unpredictable.

Your own judgment, over time. This is the risk I take most seriously and the one with the least evidence either way. The unpleasant part of writing — sitting with a bad paragraph long enough to work out why it is bad — is where the skill is built. An always-available alternative to that discomfort is an always-available way to not develop.

The disclosure question

The rules are less dramatic than the discourse. Amazon's self-publishing platform, for instance, requires you to disclose AI-generated text, images, or translations, while AI-assisted content — material you wrote and then edited with AI help — does not require disclosure. The distinction is between who produced the words and who produced the help.

Elsewhere it varies: many literary magazines prohibit generated submissions outright, competitions increasingly require declarations, and traditional publishers ask. The practical advice is boring and sound. Know which side of the generated/assisted line your process falls on, be able to say so if asked, and do not claim a book is entirely your own writing if substantial passages were not.

There is also a copyright dimension worth knowing rather than assuming. Purely machine-generated text sits on uncertain ground in several jurisdictions, because copyright protection has historically attached to human authorship. If your book's text is substantially generated, what you own may be less than you think.

The honest summary

These systems are extremely good at producing competent prose and have no capacity whatsoever for knowing which prose is worth producing. That is not a temporary limitation waiting on the next model release; it is what an averaging process does.

Which puts the useful role fairly precisely. They are a tireless first reader, a generator of options you will discard, a continuity clerk, and an escape hatch from a blank page. They are not a co-author, whatever the marketing says, because co-authorship requires having something you want the book to be.

The writers I have seen get the most from this are the ones who already had a clear voice and used the tools to remove drudgery from around it. The ones who got least were trying to use fluency as a substitute for having something to say — and fluency has never been the scarce resource. There was never a shortage of competent prose in the world. There was a shortage of people with something specific to tell you and the stubbornness to find the right words for it.

That shortage is untouched.