P1-5.1 What Does Learning Change?¶

Section ID: P1-5.1 Version: v2026.07.20

Chapter 4 organized real-world problems in terms of input, output, feature, representation, and parameter so that a model could handle them. Now we move one step further and ask: when we say a model learns, what inside the model actually changes?

In AI, learning is a broad concept that means performance on a task improves after experience data. Within that broader idea, training is the more concrete procedure in which the model sees examples and adjusts internal values to reduce output error. Those internal values mainly refer to parameters, especially weights and biases.

In Part 1, this section fixes the baseline distinction among learning, training, and fitting. The baseline meaning of parameter, weight, and bias was introduced in 4.3, and the execution perspective of inference is developed in 5.2. Here the focus is narrower: separating what changes from what procedure causes that change.

At the same time, this topic can easily be read too broadly. Saying that a model learns does not mean it understands the world the way a person understands experience. In this section, we keep the terms narrower:

learning = the broad idea that task performance improves after experience data
training = the procedure that adjusts internal model values to fit data and goals

Three Standards¶

The useful first distinction is:

Here we keep the distinction: learning is the broad result, training is value adjustment, fitting is an execution form, and inference is the usage stage.

Why Is It Called Learning?¶

The term learning in machine learning does not mean a model gains understanding or consciousness like a person. It is used in a narrower technical sense.

The Deep Learning book introduces Tom Mitchell’s definition: a program learns if its performance on a task improves through experience. Three words matter in that definition.

So we use the word learning because the model sees experience-like data, changes internal values, and improves on a defined task by a measurable standard.

experience data enters
internal model values are adjusted
task performance improves
so we call it learning

This resembles human learning only in one limited way: past experience affects later behavior or judgment. But human learning also involves understanding, memory, context, purpose, bodily experience, and social interaction. Machine-learning learning is usually defined much more narrowly, in terms of data, goals, performance criteria, and internal value adjustment.

In this book, when we say the model learns, we mean:

the model sees data,
adjusts the internal values used in output computation,
and does so with respect to a defined goal

It Is Safer to Distinguish Learning and Training¶

In research and technical writing, learning and training are not always treated as a perfectly strict pair of opposites. A learning algorithm may include the training procedure, and everyday technical writing often moves naturally between train a model and learn from data.

Still, the two words have different centers of gravity. Learning often points to the broader phenomenon or problem setting in which performance improves after experience. Training more often points to the concrete procedure of adjusting model parameters using data.

Since Korean often translates both as one word, these two levels can easily mix. In this book, we keep them apart like this.

In that sense, learning is the broader result and training is a representative procedure inside it.

learning: what got better?
training: by what procedure were the values changed?

It is safer to read this distinction as a flow rather than as a rigid classification table. Read learning broadly, read training as a representative procedure inside it, and place inference as the next stage that uses the result.

The same scene can be worded differently depending on which level is being discussed. Saying the model learned support-message classification usually points to the result side: performance on new messages improved. Saying the model was trained on support data points more to the procedure: labeled examples were used to adjust internal values.

For example, the model learned customer-inquiry classification is a broad statement. More precisely, it can be separated like this.

training: past inquiries and labels were used to adjust model parameters learning: as a result, classification performance on new inquiries improved inference: the adjusted model predicted the type of a new inquiry

So the learning in the title of this section is a broad introductory expression. When the body discusses internal value adjustment, it uses training or learning procedure more precisely.

Experience and Signals Differ by Learning Type¶

The support-message example so far is closest to supervised learning: the input sentence and desired output label are given together, and the model is trained to reduce the difference between prediction and label. But not all AI learning has that shape.

So the explanation training reduces the difference between prediction and label is a simplified entry point for supervised learning only.

In reinforcement learning, the center shifts from correct labels to actions and rewards. In unsupervised learning, the center shifts to the structure or representation in the data rather than a human-provided answer.

Deep learning is a different axis. It is not a learning type in the same sense. It is an approach that uses multilayer neural networks to learn representations, and it can be combined with supervised learning, unsupervised learning, self-supervised learning, or reinforcement learning.

learning type: by what experience and signal does performance improve?
deep learning: by what structure are representations and parameters learned?

Here, the supervised-learning example is used first to build the intuition that internal values are adjusted. The finer differences among reinforcement learning and deep learning return in P1-8, P1-9, Part 4, and Part 5.

Section 5.1 explains the intuition behind learning and training. It does not go into the formulas or implementations of loss function, gradient descent, backpropagation, or optimizer. Those return later in Part 4 and Part 5.

It also does not explain inference in detail. Inference is handled separately in 5.2. Here we mention only enough to distinguish it from training: values changed during training are used later when the model is run.

Adjusting Internal Values Through Training¶

• Understand training as the process of adjusting internal model values.
• Distinguish learning from training.
• Avoid mixing supervised learning, unsupervised learning, reinforcement learning, and deep learning as if they were the same category.
• See how parameters, weights, and biases connect to learning.
• Understand loss as the signal that guides learning.
• Distinguish learning from simply storing all data as-is.
• Prepare the boundary among training, fitting, and inference.

Training Is the Process of Adjusting Criteria from Examples¶

Google’s Machine Learning Glossary explains training as the process of determining the ideal parameters that make up a model. During training, the system reads examples and adjusts parameters gradually.

Continuing the customer-support classification example:

At first, the model may not know which clues should connect strongly to which outputs. Training adjusts those internal criteria little by little.

example input
-> model prediction
-> comparison with desired output
-> internal value adjustment in the direction that reduces the difference

The numeric form of that difference is loss.

large loss = the model output differs a lot from the desired output
small loss = the model output is closer to the desired output

What Changes Is Mainly the Parameters¶

Section 4.3 defined a parameter as an adjustable internal value the model uses to turn input into output. We can now add one more sentence:

training is the process of changing parameters

Google’s glossary explains parameters as the weights and biases that a model learns during training. In simple terms, weights determine how strongly some value influences another, and training searches for weights that better fit the task.

In a simplified support-message classifier, it may look like this:

This does not show the literal inside of a real model. It is a simplified way to see what learning changes.

One beginner confusion appears here: people hear the model changed and imagine the whole structure was rebuilt from scratch. At the introductory level, it is safer to understand that the overall model framework stays in place while internal values change inside it.

Bias Changes the Default Position¶

If weights adjust the influence of input clues, bias can be read as the value that shifts the default position of the output when input clues are weak or absent.

Suppose most support messages in a service are delivery-related. Even when the model sees little evidence, it may tilt slightly toward delivery. That default tilt can also become something learned.

weight: how strongly should a clue matter?
bias: where should the default output position begin when evidence is weak?

This is an intuition mainly useful for linear models and neural networks. Not every model exposes internal structure in exactly this way.

Learning Is Not the Same as Memorizing an Answer Sheet¶

If we reduce learning to memorizing data, we misunderstand machine learning. Of course, large models can remember or reproduce parts of training data, and that matters later in discussions of copyright, privacy, and security.

But the main goal of learning is not to store the training data exactly. It is to find relationships that can also be used for new inputs. The Deep Learning book distinguishes fitting training data from finding patterns that generalize to new data.

Using the support-message example:

For example, these two sentences differ in wording but may carry similar business intent:

I want a refund.
Can I cancel the payment?

A learning-based model tries to adjust itself so that different expressions can still lead to the same output when the training objective supports that pattern.

The safer interpretation is:

a learned model is adjusted to compute relationships between input representations and outputs
according to training data and the learning objective

Learning Moves in the Direction That Reduces Loss¶

Any learning process needs a criterion for what counts as better. The loss function quantifies that criterion.

For example, suppose a classifier predicts like this:

In real training, the model usually uses more detailed scores or probabilities rather than simple right-or-wrong judgments. Once loss is computed, the training procedure adjusts internal values in the direction that reduces it.

Google’s glossary describes gradient descent as a mathematical technique for minimizing loss by repeatedly adjusting weights and biases.

The important point here is not the formula, but the role.

loss: a signal of how bad the current prediction is
training procedure: a way to change internal values so that loss becomes smaller

Training Is Repeated, Not Finished in One Step¶

Training usually does not end with one example. The model looks at many examples repeatedly and adjusts itself little by little.

initial model
-> check a batch of examples
-> compare predictions and labels
-> compute loss
-> adjust parameters
-> check examples again

If loss decreases, we can say the model is fitting the training data better. But fitting training data too well is not always good, because overfitting may make the model weak on new data. That broader issue returns later in Part 4. Here the point is only that internal values are adjusted repeatedly during learning.

In Deep Learning, Representations May Change Too¶

Section 4.3 described representation as the transformed form that makes original data easier to compute over. In more classical machine learning, people often design features and the model learns parameters on top of them.

In deep learning, that boundary becomes less sharp. As input passes through several layers, the network builds internal representations, and learning adjusts the weights used to create those representations as well.

In a simplified support-message example:

sentence
-> word or token representation
-> internal vector representation
-> message-type output

So in deep learning, learning may change not only the final output criteria, but also the internal values that create intermediate representations.

That is why deep learning often makes the question does a person design the features directly? less central than can the model learn useful representations?

Here, first keep the view that intermediate representations can also change. The way this connects to hidden representations, embeddings, and the role of deeper layers returns in P1-3.3 and Part 5.

Fitting Also Appears as a Practical Term¶

When you use machine-learning tools, you often see fitting together with training. scikit-learn describes fit as the operation that matches an estimator’s state to data, and a fitted estimator as one that holds the information needed for prediction in internal attributes.

In this section, we separate the terms like this.

In practical code, training is often triggered through APIs like model.fit(X, y). In this book's conceptual explanations, learning is used as the broader term, while training is used for the procedure that adjusts internal values.

The Word Parameter Still Has Multiple Levels¶

As noted in 4.3, the word parameter changes meaning with context. Google’s glossary uses it for weights and biases learned by the model. By contrast, the scikit-learn glossary also uses parameter for configuration values passed to an estimator constructor.

To reduce confusion, this book separates them like this.

So when this section says learning changes parameters, it means the first sense: model parameters. Generation settings such as an LLM's temperature are not internal parameters obtained by learning.

Learning and Inference Are Different¶

Training and inference are often mentioned together, but they are not the same process. Google’s glossary distinguishes training, which finds good weights, from inference, which uses learned weights to produce predictions.

The boundary to keep first is:

Real systems can include online learning, continual learning, caches, retrieval, and tool calls, so the full picture can be more complex. But this baseline distinction is still necessary:

training changes values
inference uses the changed values

Section 5.2 looks separately at what inference actually executes.

A Small Example Again¶

Suppose we build a support-message classifier.

goal: classify the message type from the support sentence
input: support sentence
output: refund, delivery, exchange, other
training data: support sentences with human-assigned labels

Before learning, the model may be wrong like this:

Learning adjusts internal values so that these differences shrink. After learning, the model may behave more like this:

But that alone is not enough. We still need to check whether it works on new sentences that were not in the training data.

The item I received yesterday was broken, and I want it resent.

If the model can handle that as exchange or reshipment, then it has captured at least some more general relation rather than only exact memorization. This judgment must be checked with validation data and evaluation criteria.

Checklist¶

• I can explain training as the process of adjusting internal model values.
• I can explain how parameters, weights, and biases connect to learning.
• I can explain that loss is the signal that guides the direction of learning.
• I can explain that learning is not the same as memorizing the training data as-is.
• I can state the basic difference between training and inference.
• I can distinguish that the word parameter can refer to model-internal values, hyperparameters, or API settings.
• I can explain that learning is not a process of injecting knowledge into a model like a human mind, but a process of adjusting internal values that turn input representations into outputs according to data and goals.
• I can explain that before asking what a model learned, we should ask from what data, toward what output goal, through what loss criterion, and with which internal values adjusted.

Sources and Further Reading¶

• Google for Developers, Machine Learning Glossary, accessed 2026-06-22.
• Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, Chapter 5: Machine Learning Basics, MIT Press, 2016, accessed 2026-06-22.
• scikit-learn developers, Glossary of Common Terms and API Elements, accessed 2026-06-22.