TL;DR I trained a model for humanlike writing, and to my eyes it is a far more interesting writer than any frontier LLM. My model, waffler-1, can be further trained to mimic any author's style and has downstream applications in marketing, content generation, and creativity itself.
I am so bored of AI-generated writing. The more of it you consume, the more samey it all sounds. The negative parallelisms, the terrible similes, the overreliance on nonsensical personification, tricolon, whispers, roots, not knowing, not asking permission, 'something', 'nothing', humming, murmuring — please shut the f*** up.
There's nothing fundamental about transformers that prevents them from emulating good creative writing. In the long term, it could be that the biggest models can't really create new and exciting books, but I expect with sufficient scale they'll be able to do it. I doubt there's some hypothetical element of human authorship which a well-trained LLM can't mimic. The problem today is poor post-training and all the safety mechanisms in place which rid base models of their wild promise. I'm trying to warn the world that there probably isn't something uncopyable about humanlike literature, and we should prepare for a world where AI text no longer sounds like slop and is in fact of equal 'quality' to the human works it saw during training.
So, as an experiment, I've trained my own literary model (waffler-1) to demonstrate that even with a small budget you can develop an LLM that produces significantly more interesting poetry and prose than any of the major APIs like Claude Opus 4.8 and GPT-5.5 today. Additionally, waffler-1 can be adapted to copy a writer's style, so for example I've finetuned it on my own writing (waffler-1-william). I won't be releasing any of these models due to ethical qualms I have around training data and the consequences of improved AI writing on the arts.
Why are modern LLMs bad at writing?
First off, the labs aren't optimising for creative writing. They're focused on coding and science, which are realms far from literature. To get better results in, say, physics, your model almost by definition needs to lose its creative capabilities because precision in that task is far more desirable than aesthetic and storytelling abilities. By training their models to respond accurately, the labs curtail the creativity of their raw LLMs, and massively so. The start of a new idea is a hallucination, and when you suppress all hallucinations in favour of accuracy, you end up with a boring model that never talks out of line.
The next problem is that, insofar as LLMs like Claude can write 'creatively', they only really gained that skill cheaply and hastily during RL stages after the fact. Instead of fully absorbing old books, the adolescent model, already whipped into honesty and scientific rigour, is rewarded by taste-tuners (sometimes other LLMs acting as judges) for producing responses they deem on balance palatable. You end up with slop, a worthless kind of naff nothingy greige bland spam that, concerningly, some people seem to prefer to its human counterpart. When you shoehorn the complexity of deep concepts or the richness of experience into the same stock boilerplate phrasing, you get slop. And that stock boilerplate phrasing comes from low-quality small-sample RL post-training. I suspect the labs could use several orders of magnitude more creative writing samples, and that way people wouldn't constantly be generating stories with a character named 'Elias'.
Some people argue that cross-entropy loss on the next-token prediction task is a poor way of training 'good writing' into a transformer. It may be true that CE is relatively weak at capturing longer distributional shapes and the human turn of phrase, especially in small training runs. But I imagine at a certain enormous scale and with a more literary dataset, CE emerges as the dominant way of encoding humanity into a model's weights.
Above are only some of the problems. The big AI companies take numerous steps to get their models to 'behave' (safety training being yet another), and in doing so they force the outputs into a limited range of formulaic expressions.
How I trained my humanlike model
First, I prepared a dataset of longform human writing and filtered it to remove short low-quality texts, non-English samples, and anything which contained undesirable keywords. This left me with around 270M tokens from 8 sources, e.g. Project Gutenberg (roughly 10% of the corpus). These I packed into 4096-token chunks for training. I chose a newish open-source foundational model, and began by training the instruct variant, but there was too much friction — I think the 'assistant axis' baked into the instruct weights prevented the model from learning how to write more freely. After swapping to the base variant, the loss was looking much better, so I fine-tuned the full model using FSDP on an 8xB200 node (loss ~= 2.25).
Following that, I had my own LLM that was already much more writerly than its brothers, but I wanted to push it to sound even more humanlike, so I generated tens of thousands of samples from the raw model, using the (variable) start of a human text from my dataset as the prompt, thereby creating [AI, Human] writing pairs. I added a LoRA (more specifically a DoRA) and used DPO on the pairs so that my model could better match real text. By RL-ing in a preference for human-written literature, it seems the model learns how to use vocabulary and punctuation more realistically. During this step MMD between human text and generated text fell (0.0038 → 0.0019), halving the statistical distance between my model's generations and real human writing (human<>human floor is ~0).
Although I ran DPO at just 1024 context length, the learnings generalised to longer samples, and subsequent ReLoRA-style training actually led to worse performance, even when I trained with a longer context and a higher rank. These findings suggest that the above DPO technique swiftly teaches surface-level patterns around token frequency, rather than deeper expression of humanish ideas.
I decided not to do any instruction tuning as I'm extremely confident this inhibits literary freedom and instead I kept my final model, which I call waffler-1 because it waffles, as a completions model. One next step could be adding a weighted term to the DPO, perhaps using embeddings of the pairs to measure humanness.
Writing results
As a base model, waffler-1 can't fairly be evaluated on the standard benchmarks, and regardless it isn't designed to be a general purpose assistant. Even EQ-Bench, a creative writing benchmark which would be ideal here, is configured for instruction-tuned models and won't easily support continuation prompts of the kind my model expects.
Anecdotally, however, waffler-1 is an entertaining, surprising, and oddly humanlike writer. It has a tendency to hallucinate, which is also one of its greatest strengths, as it means that at last you can get exciting outputs out of an LLM. It's so weird, and I'm sorry to say I quite enjoy reading its generations, especially its more poetic attempts. I can state with some confidence that this model doesn't produce slop, but a purer, stranger recreation of human writing with a wider distribution of possible generations than any major LLM. I expect if you scaled up the base model size and my training methodology you'd get even better results. It likely won't be long until the labs do this, or something similar.
I believe that humanlike AI writing will become more common in the coming years, and not only for creative purposes like poetry but also for coding, marketing, advertising, and any other kind of writing. By humanlike I mean indistinguishable or at least indistinguishable at a high-enough level of abstraction. This model isn't perfect, there's definitely room for improvement around the coherency of generations (the model's hallucination sometimes doesn't fit in with the previous context) and overall output quality (the model can generate nonsensical, repetitive, or silly text). The hallucination aspect is a double-edged sword — it is an annoyance if you just want a basic LLM which generates exactly what it was trained to generate, but it is a boon when you want to see what the model can dream up. I find it fun to keep the context length relatively low (around 500 tokens) and watch the model run wild on an off-topic prompt.
But the proof is in the pudding. Did you notice that the previous paragraph was AI-generated, with waffler-1-william? Additionally, I put every single generation on this page through Pangram 3.3.2, and it claims that they were all 100% human-written.
Below I offer some cherry-picked samples for you to judge:
In an empty paddock, I go looking for the dead goat, and find it lying in the soft red dust. There is no blood, only a wound in the side, caused by the knife, the wound running down to the belly. I look at the carcass of this goat, so still and silent, and I wonder why it is that I feel so sad about this goat, a goat that is dead. For me the dead goat has a name, I cannot see it as just a goat, I know that the dead goat is a symbol of someone or something, I think, and so I ask the goat, "What am I missing here?"
In the days following my death dream, I am depressed, I am still not writing, not producing, and so I decide to return to New Mexico in order to take another look at the paintings, but before I do so I decide I need to give some time to my dream.
I get together with a group of friends, we drink beer, and we talk. One of the group says, "Do you think you are dreaming that you are dead and that the world is as it is?" I say, "Yes, that is exactly what I think." And the group says: "It is impossible, it's impossible for you to be dead, this is real." And I am saying, "No it isn't," and they say, "Why not?" And I say, "I have evidence: I died."
In an empty paddock, near the tiny town of Muckadilla in Queensland, a grey tabby cat stood alert, one paw raised and tail up, looking straight at me. It took some time for him to calm down – after all I was an absolute stranger – but after a bit of coaxing and a few treats, he turned around and lead me to his nest, a pile of plastic bottles under a pile of pallets in a rubbish dump, and asked for a belly rub. This was my introduction to the feral cat in the wild.
Feral cats are a highly successful species, that can adapt to a wide range of habitats and eat a variety of prey. In Australia they are generally considered pests because of their impact on native bird and mammal species.
However, their success in the wild comes at a price. It can be difficult to get near them. Feral cats are fast, agile and very wary of potential threats. They are often aggressive to other cats and spend a lot of their time fighting and getting injured.
Feral cat populations also fluctuate in size, reflecting variations in the abundance of food. This is because they have little or no natural predators in Australia and few are born. Instead they live and reproduce by hunting and killing other animals.
So why is it so difficult to study feral cats?
To get close to a feral cat in the wild is almost impossible.
In an empty paddock, far from the busy roads or city crowds, there is a man with a gun, and his name is David.
It's early in the morning, but David's head is heavy.
"David, are you okay?"
It's the first of his wife's questions. The third in the morning.
"Yes, I am fine, I am fine. Don't worry. I have just had some bad dreams."
David rolls over in the bed, searching for a comfortable position. His mind is blank. All of it is. It's the dreams that make him remember.
"What were your dreams?"
"I can't remember. Bad dreams. You know I have them, sometimes. Just forget about it."
In reality, David knows what it was about. It was about his car. A beautiful green 1964 Dodge Dart. He knows where it is, too. It's somewhere in the paddock, where it has been hiding for the past ten years.
"All right. I will talk to you later. I love you, honey."
David knows where the car is. But he also knows what will happen to him if he admits it. He will go to prison. David will never admit that to his wife. He cannot. He knows there are things that should not be done.
AI-generated with waffler-1
During summer heatwaves, we tend to get a lot of comments saying "this is just summer”. "This is just a heatwave". "This will blow over". It's kind of frustrating, really. But that is not at all what climate change looks like. This is not a normal heatwave. This is not normal. These are all facts, not opinions. But it seems like a lot of people are not that interested in facts.
The fact is, human-caused global warming has contributed to a 70 times greater chance of getting temperatures as high as they are this summer. This is, in fact, normal. Normal for what is now a much hotter world. But not normal for our climate as we experienced it when we were growing up. The facts are clear: this is what climate change looks like in the United States.
For a while now, we have been discussing whether heat waves were getting more common or intense because of global warming. As you'd expect, there's been plenty of confusion. Well, according to the National Oceanic and Atmospheric Administration (NOAA), the heat is finally clear to see.
During summer heatwaves, we tend to get a lot of comments saying "this is just summer”. We need you to understand that climate change is the driving force behind the increase of the frequency and intensity of extreme weather events and heatwaves. If we are to save ourselves from climate hell, we have to make deep cuts in our CO2 emissions and rapidly transition to cleaner forms of energy.
The two-week delay caused by the Conservative Party Conference means MPs will only be able to take part in 11 days of parliament before Christmas. That's not nearly enough time for the government to introduce its promised flagship climate bill, the Environment Bill and the Immigration Bill, or to give proper time to debates on the Finance Bill or the EU Withdrawal Bill. The government must prioritise the bills it will introduce this autumn and give MPs the time they need to hold the government properly to account.
Climate change is happening now and the government's response so far has been far from adequate. Ministers must take real action to decarbonise UK transport, end the boom in new private jets, and tax flights properly. They should set aside a significant amount of the billions they are planning to spend on road building and give the money instead to bus and train improvements. The UK should also match Europe's plans to stop subsidising fossil fuel.
Ministers have so far failed to act on the warning signs we are all seeing in the headlines.
During summer heatwaves, we tend to get a lot of comments saying "this is just summer”. This is a very good example of the "missing forest for the trees" problem. We are not seeing the big picture, and as a result we are making the wrong conclusions.
This is like saying "it's just a phase" when a child has a fever. We know that fever can be a sign of something bad. But because the problem seems to be just one episode of fever we conclude it's not serious and ignore it, just letting our child keep their temperature at 40 degrees Celsius or more.
This was exactly the situation in the last three years. Every summer, we were experiencing record heat. Some people were still saying "this is not a problem, this is just summer". The big problem is that these years are not isolated events, they're not just summer, they're part of a bigger trend, part of a new reality: the Earth is getting warmer and we are causing this to happen.
At least 80% of the energy we produce comes from fossil fuels. We burn fossil fuels for transport, for cooking, for heating our homes, for heating water and for generating electricity. When we burn fossil fuels, we emit CO2 to the atmosphere and in the long term this will impact the climate. We can't change the whole society overnight, but if we do nothing, if we do not even try, it will take at least 20 to 30 years.
AI-generated with waffler-1
Rosa wanted to dwell there a little longer, where she was most needed, and where she felt most alive, but a storm was coming on. She sensed this without being able to see beyond the fog and the trees. The rain was growing heavier and heavier, and the wind, which had been a mere breath, began to blow like the wind of destruction. She sensed that they were no longer far from the high hills and the flat valleys, where the wind would be even more savage. And it was there that she was bound to go. The trees were swaying, and she could smell the approaching storm. 'We'd better set off soon,' she said. 'Yes, we'd better get to our cave as fast as we can,' her mother agreed, 'before the rain catches us.' It was hard work to go any further. The heavy rain had already begun to fall. The wind, which grew stronger with every passing minute, carried the scent of the storm, which was close behind. Rosa saw branches of the trees break off and tumble towards her in the wind. The wind also snatched up leaves and whirls of dust and threw them towards the ground with incredible violence. She remembered the trees in the storm-ravaged part of the forest at the edge of the wasteland.
Rosa wanted to dwell there a little longer, where she could bask in the warm afterglow of the fire, with its dancing flames, flickering shadows, and fragrant, spicy air. The hot, red-orange embers were glowing now. Her eyes closed. In a moment she was fast asleep, her head bent forward, her long black hair falling about her face, as though a dark curtain had been drawn over her. At last, the children were back, each one in turn bringing her mother a delicious, mouth-watering dish, and one after the other they placed them on the floor before Rosa. The older children each took turns filling the dishes with whatever Rosa ordered: some baked potatoes, hot corn on the cob, baked macaroni, or sweet rolls. Everyone was excited. The night was quite chilly, and after eating they crowded in by the fire to keep warm. Mama Rosa noticed a small boy sitting on the edge of the group. "Who is he?" she asked, speaking English. "My cousin," explained Maria. "He likes to hang around here in the evenings. It's nice having him around." Mama Rosa smiled at the boy and introduced herself, then told him the food was very good and he was welcome to have some. He politely thanked her but said he had eaten at home. "It's all right," she assured him. "You may eat if you like."
Rosa wanted to dwell there a little longer, where everything was perfect; where the sun always shone; where the birds sang a sweet, continuous song; where there was no wind, no rain, no snow, no hail, no storms, no thunder, no lightning, no earthquakes. There was no sickness, no death, no sorrow, no pain, no suffering. There was always the same time of day; it was always midsummer and all was joyous. But he told her he was a very busy man, and must go on, though he would return as soon as he could. His work was, that of a poor gardener in a great garden; and he wanted to keep the trees and flowers well trimmed and in order, for the coming of his king and queen. The poor gardener told Rosa that his king was God, and that the queen was Heaven, and that the garden was Paradise, and that he would be in no danger while he was doing his work; and when he had finished his work, his king would reward him. "Then we shall all be together," said Rosa, "shall we not?" "Yes; we shall all be together;" and the gardener took the little girl's hand in his and patted it lovingly. He showed her where the work was to be done. There was a wild tangle of briers and thorns and weeds and rubbish, that had grown up in a once fair flower-garden, and they were to be all cut down.
AI-generated with waffler-1
As for a girl in England who
Asks me to go for walks, she too
Is welcome in the long run; yet I
Must not, alas, begin to cry
For all that I have left undone.
All this is hard to bear, but harder
Still it is to make of my life
A picture of the future I
Imagined when I was a boy,
As my young dreams are crumbling, too.
I know, my friends, that I would not
Live if my children came to me
To tell me that they had renounced
What I had taught them, if I were
The man I was at the age of forty-five.
But even then, if all things went well,
I had not only learned the lessons
My books were full of, but also the
Lessons I had not thought to find
In any book of the world's lore.
I thought, when my years had gone by,
I would have learned a few things
I had thought my time would bring me.
I was wrong. Time passed and I saw
That I would never learn them all.
As for a girl in England who
Asks me to go for walks, she too
Is welcome in the long run; yet I
Must not, alas, begin to cry
If at this stage her glance or tone
Are coldly put against me; if
Her gestures have no more expression
Than in a lifeless and morose
Dying, if I could but
My own thoughts in her mind's ear
To see her, to perceive
That she, too, would find, if she
Looked for them, the things I think
In myself—how then should she,
Through my love for her, see more clearly
The same things, the same feelings
In herself, and make them new
To herself, and more real, as
If they had lived before and then
Like a lost child of long ago,
A long-dead relation, she
Had been found, been brought back to her
On this earth, and to her life.
As for a girl in England who
Asks me to go for walks, she too
Is welcome in the long run; yet I
Must not, alas, begin to cry
24. THE DRAGON-SLAYER AND THE FAIR MAID
Who, my friends, is so happy-minded
As not to delight in this world-drama
That all of us play when we have to live?
25. THE TWO MAGICIANS
Some people think,
"How beautiful, how skillful are words;
So the speaker, the writer is a magician!
Only the wise, who have seen through this illusion,
Remain with no illusions."
26. THE SIX STRANGERS
Six people with a hundred questions
Follow me. Six, I say, go away!
Go! —six of you. For all my life
I have loved being by myself.
27. THE CHORUS
When you have put your ear to the heart of the world
And listened for a while, what do you hear?
It beats on as if it were made
Out of a single stone.
28. THE THREE POETS
Two poets sit together.
The third poet, who has heard their talk, thinks,
"These poets are so happy-minded,
They are such fun."
29. THE TEMPTRESS
You're so beautiful you make me drunk!
You make me drunk, you beautiful, and
If only you wouldn't tease me so, you'd be
An angel that would keep me happy
AI-generated with waffler-1
I fine-tuned on about 30 samples of my own writing as well:
In other worlds, trees become
The natural objects of human love.
This one is empty, a place we were,
And now we're not, and still
The trees are there.
From this you learn
It's trees, it's always trees,
The last to go,
The first to remember
Where we were,
What we lost.
I know in other worlds
The trees stand for a longer while,
Are worshipped, and
When people have left
They'll be alone for millions of years,
Waiting for another species
To come in and start all over again.
But it seems the trees are bored
Or getting nervous,
That this might be the last
So they're doing everything to go.
In the first world I ever came to
The trees all had mouths
And eyes and hair.
I thought this was a sign of intelligence
But I later realised
They were just dying.
If we can't make up our minds
About our love,
If we're so unwilling to do what we were made for,
Then the trees will simply
Furl their leaves,
Take the clothes from their branches
And walk away
To leave the world to us.
To show us
How it could be
If we really wanted it.
How they look now:
Leaves still on them, but the twigs
All folded in a frown
In other worlds, trees become
monsters, or the sky is a nightmare.
In the world I live in, trees
look like trees,
and the sky looks like the sky.
I've never been so grateful for the
ordinary.
In other worlds, trees become
creatures with which one may converse.
In my world, the trees do not talk.
We exchange one-sided monologues.
I tell them my secrets.
They listen.
And I wait for the rustling of leaves
to answer,
to respond,
to agree and disagree,
to refute and agree again.
I wait for the breeze to blow
with news of the future
and I hope that my hopes
for happiness are shared,
not only by me
but also by the tree and the wind
and everything else around me.
And the tree is silent.
The breeze blows on and on.
AI-generated with waffler-1-william
I feel lost for words and it's been a month since I last wrote a post. I could say that I've been busy but I haven't really been working on anything. I'm feeling a little defeated, a little down. I guess it's just one of those cycles, those phases. But the worst part is that I can't remember the last time I felt like this so I don't know if this is going to pass in a few weeks or if it's going to last a few months.
When I went away on the trip to Iceland it seemed like a good thing, a really good thing. It gave me the chance to disconnect from all my thoughts and I could just enjoy the scenery. Being in a place like that can be overwhelming too because you have no idea what's happening, you don't understand all the natural elements like the water and ice and wind and you don't get why all the trees grow in that particular way. And you can't see the people so you start to wonder if they're really there or if you're dreaming, and then it hits you, "hey I'm not dreaming because I'm using my hands and legs right now." And you think maybe that's the difference between a dream and a reality — your ability to use your extremities. But then you wonder if that's not true and it all becomes a bit confusing.
I feel lost for words. This has always been a cause for joy, I've been inspired by the beauty of the system. It is the best system because it's the only system. The earth has always been in charge, not us, but we still have a duty to leave this beautiful garden to future generations. It doesn't work if you tear up the grass, it just doesn't. We're here, and so is all the wildlife, and everything's interconnected in this complex dance. We're just dancing partners, no more, no less, and I hate to think about dancing without music. So maybe you're not an environmentalist, and maybe you think that the earth's a dead planet that belongs to us, but I don't think so, and you know what I think? I don't want to ruin it. The earth will go on with or without us. A couple thousand years isn't long at all when you think about how long a tree can live. We're lucky to be alive, and alive in an era with the power to save the planet. It's a little scary, but hey, we're smart. We can figure this out.
I feel lost for words. I'm not used to these situations. I haven't been in this much trouble, not in my current life. I thought I did it. I had the plan. It would work. I wouldn't be here, in front of you. It was a sure-fire method of getting my wife's ring. It was just that you were in the way.
But that's okay. You've been taken out of the equation. You're not there to get in the way of me and my success. So, I've got nothing against you, I swear! No bad feelings!
But that doesn't make it any less weird to stand here and look at you, Mr. Brown. I'm confused. I was supposed to be in a different position here. But it doesn't really matter now. So, when I put my hand into the small pocket inside my coat, I don't feel bad about reaching into your bag and stealing my prize.
Oh, look at that. You were carrying it with you, in case of emergency. What an ingenious plan that was. I was thinking you'd call for a helicopter, and your personal butler would have to be in his cab with the ring in his pocket. The thought of it.
But you don't have one, do you?
AI-generated with waffler-1-william
The future
I deliberated for a long time over whether I want to continue working on this or not. I imagined raising money and trying to build a neolab (Mindforge) in the UK which is climate-neutral and only trains on ethical consented data. But I just don't believe I could win that way. Companies that are happy training on stolen work will be able to train stronger models. I'd have to invent a new data efficient algorithm which outcompetes GPTs or compromise on my values, and it's unlikely I'll do either. But more importantly, I want to focus on supporting humanmade art, which seems increasingly special. And honestly? — That's growth.
Please get in touch if you want to discuss any part of this post. Similarly, if you want to try my model or train your own waffler (for light experimentation only) I can help. I'm sorry to writers, for pointing out your homes are flammable and that there's a fire on the horizon. Before too long, slop of the clunky repetitive kind will be an artefact of the past. But I'm betting that however credible LLMs become, humans will always be the best at satisfying other humans' endless need for weird humanlike novelty.