Large language models

What an LLM is

A large language model is a machine learning model trained on a very large amount of text to predict the most likely next word (more precisely, the next token, a word or part of a word) in a sequence. By repeatedly predicting the next token it can answer questions, summarise, translate, write and explain code.

Outline of an LLM system:

1. A user's prompt (query) is split into tokens.

2. The model uses its learned patterns, plus any context given (the conversation so far, documents provided), to predict a response one token at a time.

3. The response is returned, often after safety filtering.

Models are trained on huge data sets using large amounts of computing power, then often fine-tuned with human feedback.

Limitations

• Context and query accuracy: the answer depends on how clear and specific the prompt is and what context is provided. Vague queries give vague or wrong answers. The model can only take account of a limited amount of text at once (its context window).
• Knowledge limitations: the model only 'knows' patterns from its training data up to a cut-off date, so it may not know recent events. It predicts plausible text rather than looking facts up, so it can produce confident but false statements, often called hallucinations, including invented references.
• Subjectivity and bias: training text contains opinions and biases, which the model can repeat. Answers to subjective questions reflect patterns in the data, not a neutral truth.

So outputs should be checked against reliable sources, especially for facts, code and anything important.

Benefits and drawbacks

Benefits: fast drafting and summarising, help with explaining concepts and writing code, natural language interface, available at any time. Drawbacks: possible errors and hallucinations, bias, privacy risks if sensitive data is entered, large energy use in training and running, questions over copyright and academic honesty.

Worked example

A student asks an LLM for three sources about a recent event and gets three convincing references. Explain why they should check them.

1. The event may be after the model's training cut-off, so it may lack reliable knowledge.
2. LLMs predict plausible text and can invent references that look real (hallucination).
3. Checking each source confirms that it exists and says what is claimed.

Answer: Knowledge limits and hallucination mean outputs must be verified.

Check yourself

1. How does a large language model generate a response?
   • By looking up the exact answer in a database of facts
   • By repeatedly predicting the most likely next token based on patterns learned from training text
   • By running the user's prompt as program code
   • By copying a single web page
   Show answer

   By repeatedly predicting the most likely next token based on patterns learned from training text. LLMs are next-token predictors.

2. What is meant by an LLM 'hallucination'?
   • A response that takes too long
   • A prompt that is too short
   • An image generated by the model
   • A confident but false or invented output
   Show answer

   A confident but false or invented output. The model produces plausible text that is not true.

3. Why might an LLM give out-of-date information about recent events?
   • It deliberately hides recent events
   • Its knowledge comes from training data that ends at a cut-off date
   • It only reads the first word of a prompt
   • It cannot process numbers
   Show answer

   Its knowledge comes from training data that ends at a cut-off date. Training data has an end date.

4. Which change is most likely to improve the accuracy of an LLM's answer?
   • Using a shorter, vaguer prompt
   • Asking the same question in capital letters
   • Giving a clear, specific prompt with relevant context
   • Removing all context
   Show answer

   Giving a clear, specific prompt with relevant context. Query accuracy depends on clarity and context.

5. Explain why an LLM's output may be biased.
   Show answer

   It is trained on large amounts of human-written text that contains opinions and biases, and it learns and can repeat those patterns.. Bias comes from the training data.

6. A model's context window holds 8000 tokens. A document uses 6200 tokens and the conversation so far uses 1500. How many tokens are left for the reply?
   Show answer

   300. 8000 - 6200 - 1500 = 300 tokens.

Practise with instant feedback Print this lesson

Sources

• Pearson Edexcel International Advanced Subsidiary/Advanced Level in Computer Science specification (2026)

Last updated 8 November 2026. Spotted a mistake? Tell us.