What is a language model? An explainer without the jargon

Most explanations of AI are either marketing or engineering. What a parent or a menahel actually needs is a working mental model — accurate enough to predict behaviour, simple enough to hold in your head. That is what this is.

The one-sentence version

A language model is a system that has read an enormous quantity of text and learned, in extraordinary statistical detail, which words tend to follow which other words.

When you ask it something, it is not looking up an answer. It is producing the words that most plausibly follow your question, one at a time, each choice informed by everything it has written so far.

That sounds reductive, and the surprising thing — the thing that took researchers by surprise too — is how much genuine capability falls out of doing that well. To predict the next word in a paragraph of legal reasoning, you have to have absorbed something about legal reasoning. Prediction at sufficient scale looks a great deal like understanding.

Why this explains everything that goes wrong

It invents things that sound right

This is the single most important consequence. The model is producing plausible text, and a fabricated citation is extremely plausible text — it has the shape of thousands of real citations it has seen. It has an author, a year, a journal, and no existence.

Nothing in the mechanism distinguishes recalling from inventing. Both are the same operation: produce what plausibly comes next. This is why "it makes things up" is not a bug being fixed but a property being managed.

It is equally confident when wrong

Hesitation in human speech signals uncertainty. A model produces hesitation only when hesitation is the plausible next text — which has nothing to do with whether the content is right. There is no internal confidence being expressed. This is why children in particular are poorly placed to catch errors: every cue they use to judge a speaker is absent.

It does not know what it does not know

Asked about something obscure or fictional, it will often produce a confident description rather than saying it does not know, because a confident description is more plausible text than an admission of ignorance. Newer models handle this better; none handle it reliably.

It has no memory unless the product gives it one

Each request is answered from the conversation it is shown. Anything remembered across sessions is the product storing text and feeding it back — a design decision, not something the model does. Worth knowing when you ask what a tool remembers about your child.

What training actually was

Two stages, roughly. First it read an enormous amount of text and learned to predict. That produces something knowledgeable and unhelpful — it will continue your sentence rather than answer your question.

Then it was shaped by humans rating responses, teaching it that answering, being useful, and refusing certain things score well. This second stage is where its manners and its refusals come from — which is worth knowing, because it means the refusals reflect the judgement of the people who did the shaping. That is exactly why a general assistant's idea of appropriate is not your community's: different people, different brief.

What it is not

• It is not a search engine. It is not consulting sources as it answers, unless the product has specifically been built to do that.
• It is not a database. There is no record being retrieved that could be checked.
• It does not understand in the way a person does, and arguing about whether that word applies is less useful than knowing it cannot tell when it is wrong.
• It is not a person, and it is not anyone. There is no one at the other end forming an impression of your child.

What follows for a school or a family

Three things, and they all fall out of the mechanism rather than from anyone's opinion.

1. Anything that matters must be checked elsewhere, because plausibility and accuracy are different properties and the model optimises the first.
2. Its refusals reflect whoever shaped it. If you want your own standard applied, it has to be applied around the model, by a system that checks requests and answers against rules you wrote.
3. Fluency is not evidence of anything. This is the single most useful thing to teach a child about it, and it does not expire as models improve.