Humans Necessary for ChatGPT and Other Language AIs – Sociologist Explains

The media hype surrounding ChatGPT and other advanced language model AI systems covers various topics, including the practical aspect of replacing traditional web search with large language models, the worry of job loss due to AI, and the exaggerated concern of AI posing a threat to humanity’s existence.

All of these themes have a common denominator: the grand models of language foretell artificial intelligence that will supersede humanity.

But the big language models, for all their complexity, are really stupid. And despite the name “artificial intelligence”, they are completely dependent on the knowledge and work of humans. Of course, they cannot reliably create new information, but there is more to it.

ChatGPT can’t learn, improve, or even keep up to date without people feeding it new content and telling it how to interpret, let alone programming the model and building, maintaining, and enhancing its hardware. To understand why, you first need to understand how ChatGPT and similar models work and what role humans play in making them work.

How ChatGPT works

Large language models such as ChatGPT work extensively by predicting which characters, words, and sentences will follow each other in sequence based on training datasets. In the case of ChatGPT, the training dataset contains huge amounts of public text scraped from the Internet.

Imagine I trained a language model with the following sentences:

Bears are large, furry animals. Bears have claws. Bears are secretly robots. Bears have noses. Bears are secretly robots. Bears sometimes eat fish. Bears are secretly robots.

The model would be more inclined to tell me that bears are secretly robots than anything else because this set of words occurs most often in its training data set. This is obviously a problem for models trained on fallible and inconsistent data sets – which is all of them, even academic literature.

People write a lot of different things about quantum physics, Joe Biden, healthy eating, or the January 6 uprising, some of which are more valid than others. How is a model supposed to know what to say about something when people are saying lots of different things?

The need for feedback

This is where feedback comes in. If you use ChatGPT, you’ll notice that you have the option to rate responses as good or bad. If you rate them as bad, you’ll be asked to give an example of what a good answer entails. ChatGPT and other major language models learn which responses, which predicted text sequences, are good and bad through feedback from users, the development team, and contractors hired to mark the results.

ChatGPT cannot compare, analyze or evaluate arguments or data on its own. It can only generate strings of text that are similar to those that other people have used to compare, analyze, or evaluate, rather than those that it has been told are good answers in the past.

Thus, when the model gives you a good answer, it takes advantage of a large amount of human work that has already gone into telling it what is and is not a good answer. There are many, many human workers behind the screen, which are always needed if the model is to be continuously improved or the content coverage expanded.

An investigation recently published by journalists in Time magazine revealed that hundreds of Kenyan workers spent thousands of hours reading and flagging racist, sexist and disturbing posts, including graphic depictions of sexual violence, from the darkest depths of the Internet to teach ChatGPT not to copy. such content.

They were paid no more than US$2 an hour, and understandably many reported experiencing emotional distress as a result of this work.

What ChatGPT can’t do

The importance of feedback is directly reflected in ChatGPT’s tendency to “hallucinate”; that is, confidently give inaccurate answers. ChatGPT cannot provide good answers on the subject without training, even if good information on the subject is widely available on the Internet.

You can try this yourself by asking ChatGPT more and less obscure things. I’ve found it particularly effective to ask ChatGPT to summarize the plots of various fictional works, as it appears that the model has been trained more rigorously on non-fiction than fiction.

In my own testing, ChatGPT summarized J.R.R. Tolkien’s Lord of the Rings, a very famous novel with only a few mistakes. But its summaries of Gilbert and Sullivan’s “The Pirates of Penzance” and Ursula K. Le Guin’s “The Left Hand of Darkness” — both a little narrower but far from obscure — come close to playing Mad Libs with character and place names. It doesn’t matter how good the corresponding Wikipedia pages for these works are. The model needs feedback, not just content.

Because large language models don’t actually understand or evaluate information, they rely on humans to do it for them. They are parasites on human knowledge and work. As new sources are added to their training datasets, they need new training on whether and how to build sentences from those sources.

They cannot judge whether news reports are true or not. They cannot evaluate arguments or weigh trade-offs. They can’t even read an encyclopedia page and only make coherent statements with it, or accurately summarize the plot of a movie. They rely on people to do all these things for them.

Then they edit and remix what people have said, and rely on more and more people to tell them if they’ve paraphrased and remixed well. If the general wisdom on a topic changes—for example, whether salt is bad for your heart or whether early breast cancer screenings are helpful—they must be extensively reeducated to accommodate the new consensus.

Many people behind the curtain

In short, the big language models are far from being examples of fully independent AI, but instead describe the complete dependence of many AI systems, not only on their designers and maintainers, but also on users. So if ChatGPT gives you a good or useful answer about something, remember to thank the thousands or millions of hidden people who wrote the words it squeezed out and taught it what were good and bad answers.

ChatGPT is by no means an independent super-intelligence, like all technologies, nothing without us.