大語言模型（LLM）如何預測下一個 token？瞭解 token 預測原理

Ever wondered how ChatGPT can finish your sentences, explain complicated topics, or write an entire story from a single prompt?

It might seem like the AI understands your question and then magically produces an answer.

But one of the fundamental ideas behind large language models (LLMs) is surprisingly simple:

They predict what token is likely to come next.

Let's understand what that actually means, how the process works, and how a simple prediction turns into an entire response.

First, What Is a Token?

We often say that an LLM predicts the next word, but that's not entirely accurate.

Most LLMs work with tokens. A token can be a whole word, part of a word, punctuation, or another piece of text.

For example, the sentence:

Machine learning is fascinating!

might be split into tokens resembling:

["Machine", " learning", " is", " fascinating", "!"]

The exact tokenization depends on the model's tokenizer.

Why not just use complete words? Because tokenization lets models handle common words, unfamiliar words, punctuation, and different languages using a manageable vocabulary.

So when we say an LLM predicts the next word, we're usually talking about predicting the next token.

How Does an LLM Predict the Next Token?

Imagine you start typing:

The sun rises in the ...

What could come next?

Maybe east, morning, or something else.

The model processes the text you've provided and estimates the probabilities of possible next tokens based on patterns learned during training.

For a simplified example, it might produce:

These numbers are illustrative, not actual model outputs.

The model can then select a token based on its generation strategy. If it selects east, the text becomes:

The sun rises in the east

But it doesn't stop there. It predicts the next token using the updated text, then repeats the process.

From One Token to an Entire Answer

Here's the important part: the model generates text one token at a time, repeatedly using the context available to it.

Your Prompt
    ↓
Process Existing Tokens
    ↓
Calculate Next-Token Probabilities
    ↓
Select a Token
    ↓
Add It to the Text
    ↓
Predict Again
    ↓
Continue Until Generation Stops

Suppose you ask:

“Explain machine learning in simple terms.”

The model generates a token, then another, and then another. Each new token becomes part of the context used for subsequent predictions.

This is how a single prompt can turn into paragraphs of coherent text.

But Where Do These Predictions Come From?

An LLM doesn't begin with a built-in list of every possible answer.

During training, it processes enormous amounts of text and learns statistical patterns in how tokens relate to their context.

For example, it may learn that certain words commonly appear together, how sentences are structured, how explanations are written, and how questions are often answered.

Modern LLMs typically use the Transformer architecture, which helps them process relationships between tokens through mechanisms such as self-attention.

During training, the model adjusts its internal parameters to improve its predictions. Over time, it learns complex patterns that help it generate useful text.

This is more sophisticated than simply looking up the most common next word. The predictions depend on the surrounding context and the patterns encoded in the model's parameters.

Let's Build a Tiny Next-Word Predictor in Python

We don't need a neural network to understand the basic idea. Let's create a tiny example using word frequencies.

from collections import defaultdict, Counter

sentences = [
    "machine learning is useful",
    "machine learning is powerful",
    "machine learning is fascinating",
    "deep learning is powerful"
]

next_words = defaultdict(Counter)

for sentence in sentences:
    words = sentence.split()

    for i in range(len(words) - 1):
        next_words[words[i]][words[i + 1]] += 1

context = "machine learning is"
last_word = context.split()[-1]

print(next_words[last_word].most_common())

Output:

[('useful', 1), ('powerful', 1), ('fascinating', 1)]

Our tiny program counts which words follow is in the training sentences.

Because all three candidates appear once after is in this small dataset, they have equal counts.

This is not how a modern LLM works internally. It simply demonstrates the core idea of using previously observed patterns to estimate what might come next. Real LLMs use learned neural network parameters to produce contextual probability distributions over their token vocabulary.

Why Doesn't ChatGPT Give the Same Answer Every Time?

If the model estimates probabilities for possible next tokens, why doesn't it always choose the most likely one?

Because text generation can use different sampling strategies.

With greedy decoding, the model selects the highest-scoring token at each step. With sampling, it can choose among candidate tokens according to their probabilities.

One setting you might encounter is temperature, which changes how concentrated or spread out the token probabilities are during sampling. Lower temperatures generally make choices more predictable, while higher temperatures generally allow more variation.

For example, when answering a factual question, you might prefer a more predictable response. When writing a creative story, you might want more variety.

Temperature doesn't add knowledge to the model; it changes how token choices are made.

If It's Predicting Tokens, Does That Mean It Doesn't Understand Anything?

This is where things get interesting.

Next-token prediction describes a fundamental part of how many LLMs are trained and generate text. But learning to predict tokens across diverse contexts can require the model to represent grammar, concepts, relationships, and aspects of the world in its internal parameters.

At the same time, fluent text isn't proof that every statement is true or that the model understands things exactly as a human does.

An LLM can produce a convincing answer and still make a factual mistake. Its output depends on its training, context, model design, and generation process.

The Big Picture

An LLM doesn't write an entire answer in one magical step. It repeatedly processes context, estimates probabilities for the next token, selects a token, and continues generating.

Remember the basic loop:

Context → Predict → Select → Add → Repeat.

That simple-sounding process, powered by enormous neural networks trained on large amounts of data, is one of the foundations behind modern conversational AI.

And the next time ChatGPT gives you a surprisingly detailed answer, you'll know the basic mechanism behind how those words appear — one token at a time.