MIMICRY: If chatbots can pretend to write like humans, we can also pretend to write like chatbots

AI annotator, or more precisely as a “senior data quality specialist”

Snippets from current UK job ads for AI annotation work give a clue as to the range of tasks involved: 

• “create responses that will form the ‘voice’ of future AI”; 
• “provide feedback to teach AI models to become more helpful, accurate, and safe”; 
• “write clear, concise, factually and grammatically correct responses”; 
• “coach an AI model by assessing the quality of AI-generated writing, reviewing the work of fellow writing evaluators, and crafting original responses to prompts”. 

If chatbots can pretend to write like humans, we can also pretend to write like chatbots.

‘If journalism is going up in smoke, I might as well get high off the fumes’

Journalists and other writers are employed to improve the quality of chatbot replies. The irony of working for an industry that may well make their craft redundant is not lost on them

Large language models (LLMs) such as ChatGPT have made it possible to automate huge swaths of linguistic life, from summarising any amount of text to drafting emails, essays and even entire novels. These tools appear so good at writing that they have become synonymous with the very idea of artificial intelligence.
But before they ever risk leading to a godlike superintelligence or devastating mass unemployment, they first need training. Instead of using these grandiloquent chatbots to automate us out of our livelihoods, tech companies are contracting us to help train their models.

As well as providing our model with ‘gold standard’ material, we help it attempt to avoid ‘hallucinating’ – telling lies

The core part of the job is writing pretend responses to hypothetical chatbot questions. This is the training data that the model needs to be fed. The “AI” needs an example of what “good” looks like before it can try to produce “good” writing.

As well as providing our model with such “gold standard” material, we are also helping it attempt to avoid “hallucinating” – a poetic term for telling lies. We do so by feeding it examples that use a search engine and cite sources. Without seeing writing that does this, it cannot learn to do so by itself.
Without better language data, these language models simply cannot improve. Their world is our word.
Hold on. Aren’t these machines trained on billions and billions of words and sentences? 

What would they need us fleshy scribes for?

Well, for starters, the internet is finite. And so too is the sum of every word on every page of every book ever written. So what happens when the last pamphlet, papyrus and prolegomenon have been digitized and the model is still not perfect? What happens when we run out of words?

The date for that linguistic apocalypse has already been set. 

Researchers announced in June that we can expect this to take place between 2026 and 2032 “if current LLM development trends continue”. At that point, “Models will be trained on datasets roughly equal in size to the available stock of public human text data.”

Note the word human. Large language models do little but produce prose, much of which is already being published on the internet. 

So couldn’t we train these models on their own output (so-called synthetic data)? Our cyborg internet – co-authored by us and our word machines – could then swell ad infinitum. No such luck. Training our current large language models on their own output doesn’t work. “Indiscriminately learning from data produced by other models causes ‘model collapse’ – a degenerative process whereby, over time, models forget the true underlying data distribution,” write Ilia Shumailov and colleagues in Nature. In other words, they go off the rails and tend towards producing nonsense. Feeding something its own effluvia leads to atrophy. Who would have thought?

• Shumailov explained to me that each time a model is trained on synthetic data, it loses awareness of the long tail of “minority data” that it was originally trained on (rare words, unusual facts etc). 
• The breadth of knowledge is eroded and replaced by only the most likely datapoints – LLMs are at their core sophisticated text-prediction machines. 
• So when your original, digital data is already biased – very English language-heavy, largely US-centric, and full of unreliable forum posts – this bias will only be repeated.

If synthetic, AI-produced data is insufficient to help improve the models, then they will need something else. 

• This is especially true as concerns spread that the much-vaunted models will stop being able to improve before they’ve ever become that useful. 
• Leading startup investment firm Sequoia has shown that AI firms will need to fill a $500bn revenue gap by the end of this year to keep investors satisfied. 
• The word machines might be hungry; the capital behind them also has an appetite.

OpenAI, the trillion-dollar Microsoft protectorate behind ChatGPT, recently signed licensing agreements – potentially worth hundreds of millions of dollars – with many of the world’s main media organizations, from News Corp to the Financial Times.
But it’s not just a question of accumulating more original words. These companies need the sort of writing that the model will seek to emulate, not merely absorb.
That’s where human annotators come in.

---

In Fritz Lang’s classic 1927 film Metropolis, the ancient Canaanite deity Moloch is reincarnated as an insatiable industrial machine. 

Workers in thrall to the machine Moloch in Fritz Lang’s 1927 science-fiction classic Metropolis. Photograph: UFA/Album/Alamy

And therein lies the ultimate irony. 

Here is a new economic phenomenon that rewards writing, that encourages it, that truly values it; all while simultaneously deeming it an encumbrance, a problem to be solved, an inefficiency to be automated away. 

• It is like being paid to write in sand, to whisper secrets into a slab of butter. 
• Even if our words could make a dent, we wouldn’t ever be able to recognize it.

It is a technology that works us, as opposed to working for us. Factory workers respond to its ever-growing demands by lunging at its dials and pulling at its levers. But they cannot keep up. The machine hisses and explodes. We then see the workers forgoing the act of feeding and walking straight into the furnace mouth of Moloch themselves.

When I first took the role as an AI annotator, or more precisely as a “senior data quality specialist”, I was very aware of the irony of my situation. 

• Large language models were supposed to automate writers’ jobs. 
• The better they became through our work, the quicker our careers would decline. 
• And so there I was, feeding our very own Moloch.
• It is like being paid to write in sand. 
• Even if our words could make a dent, we wouldn’t ever be able to recognize it
  

Indeed, if there is anything these models can achieve quite well, it is the sort of digital copywriting that many freelance writers perform to pay the bills. 

Writing an SEO blog about the “internet of things” might not take much research, pride or skill; but it usually pays far better than poetry.
Working for an AI company as a writer was therefore a little like being told you were going to be paid a visit by Dracula, and instead of running for the hills, you stayed in and laid the table. But our destroyer is generous, the pay sufficient to justify the alienation. 

If our sector was going up in smoke, we might as well get high off the fumes.