P4-8.2 Baseline¶

Section ID: P4-8.2 Version: v2026.07.24

In P4-8.1, the discussion examined what model families should be raised as candidates. Now, instead of immediately grabbing those candidates in order of complexity, it moves to the question of setting the starting point of comparison first.

In this problem, what is the simplest standard that must be beaten first?

That question is exactly the starting point of the baseline.

The baseline is often understood as a temporary model with low performance. In practice, however, it is much more important than that. A baseline is the floor line of comparison that checks whether a complex model is really creating meaningful improvement.

In both academic and practical contexts, a baseline is closer not to a good model, but to the minimum standard that makes comparison possible. In other words, without a baseline, even if a performance number looks high, it becomes hard to tell whether that is because the problem is easy, because the data are biased, or because the modeling actually helped.

This Section explains the meaning and role of the baseline. Later Sections continue the current context through this handle, and the criterion for what the improvement of a complex model should be compared against reconnects through this Section and the concept glossary.

The perspective that must be fixed here is the following one sentence.

A baseline is not simply a weak opponent that must be beaten, but a reference line for reading the meaning of a score.

The order of baseline comparison is fixed briefly as follows.

To actually set up a baseline, two things are needed together.

1. a methodology for deciding what should be used as the simplest comparison standard
2. minimal theory that explains why comparison with such a simple standard must come first

This Section handles why it is needed first and what must be fixed first, while the following P4-8.3 supplementary learning handles with examples and exercises what representative baselines should be set and how.

Scope Of This Section¶

This Section answers the following questions.

• Why is a baseline needed first?
• What kind of illusion can occur if there is no baseline?
• Why must the baseline and candidate model be compared under the same conditions?
• As preparation for reading the practical procedure of setting a baseline, what should be grasped first?

This Section first closes what the improvement of a complex model should be compared against. The operational perspective of benchmarks and leaderboards, and the big picture of statistical-test-based model comparison, are reorganized again in the supplementary learning of P4-9.3, while the actual hyperparameter comparison procedure continues directly in P4-9.2.

Goals Of This Section¶

• You can explain a baseline as the comparison standard that is set before a complex model.
• You can explain why accuracy illusions and mean-prediction illusions occur when there is no baseline.
• You can explain why tools such as DummyClassifier and DummyRegressor are educationally useful.
• You can adopt the perspective that a good model is not simply one with a high score, but one that creates meaningful improvement over the baseline.
• You can explain why the next Section, P4-8.3, separately reinforces how to actually set a baseline.

Learning Background¶

In P4-8.1, the discussion set up a candidate family of models. But setting up candidates does not mean comparison starts immediately. Comparison needs a starting point.

• Even if there is a candidate family, without a comparison standard it is hard to distinguish improvement from illusion.
• Even if there is an evaluation metric, it is hard to know whether the score comes from an easy problem or from the model.
• Even if preprocessing and feature selection were done, the experiment floats in the air if readers do not confirm whether it is better than a simple rule.

So in the curriculum, this Section plays the following role.

So if P4-8.1 dealt with what should be raised as a candidate, P4-8.2 deals with by what standard should this candidate be compared.

The first three questions that must be held onto in this Section are the following.

• Is this a problem that can be matched to some degree without looking deeply at any features at all?
• Is the model I built really better than that easy standard?
• If it improved, on what metric did it improve?

One more thing must be attached here. In classification problems, the mere fact that the score went a little higher than the baseline is not enough. Only when readers place the confusion matrix and representative error cases beside it and inspect did misses decrease, did false alarms increase, and does it catch important minority cases better does baseline comparison come alive. In other words, a baseline is both a numeric comparison table and a reference line for error interpretation.

If this is translated into a customer-churn case, the baseline appears not as one score, but as the branching point for reading did the current model really go beyond the easy standard.

flowchart TD
  A["churn dataset"]
  B["baseline model<br/>always stay or dummy rule"]
  C["candidate model<br/>uses behavior features"]
  D["same metric<br/>accuracy / recall / F1"]
  E["error check<br/>missed churn cases"]
  F["meaningful gain?<br/>keep or rethink"]

  A --> B
  A --> C
  B --> D
  C --> D
  D --> E
  E --> F

One more thing needs to be remembered separately. Baseline model and baseline reference meet in the same context, but they are not exactly the same object. The former often means the simplest predictor, while the latter can include the comparison frame that places recent results next to the usual range. This Section separates these two meanings so that even when the reader writes project-retrospective documents in Part 6, the wording does not become confused.

This Section reads them in the following way.

The two are different objects, but they share something too. Both are comparison criteria that keep readers from viewing a number alone without interpretation.

So the first confirmation items that must be left in baseline comparison are the following four.

If the same criteria are moved into a recording structure, they become even clearer.

The core of this flow is that the baseline should be placed after selection and before tuning. First, candidate models are set up. Then those candidates must be comparable with the baseline. Only after that is it natural to attach tuning and algorithm-specific expansion.

Main Learning Content¶

What Must Be Fixed First Before Setting A Baseline?¶

The actual procedure for setting a baseline is examined separately in the next Section, P4-8.3. Still, even in this Section, the preparation items that must be fixed first should be held clearly. A baseline is not a rule that suddenly appears. It can be set only after what problem is being solved, what counts as one sample, and what score will be used for comparison are fixed first.

That means the first stage of a baseline is not what rule should be chosen immediately, but what comparison is being attempted. In the next Section, these preparation items are used to examine how representative baselines are actually set by problem type.

Why Is Some Theoretical Knowledge Needed For A Baseline?¶

Advanced mathematics is not needed first to set a baseline. But a minimal theoretical background is necessary. Only then can the reader explain why a baseline is not merely a roughly made weak model, but the comparison standard that must come first.

The common point that becomes clear in the directly checked documents is obvious. scikit-learn explains DummyClassifier as a simple baseline to compare against other more complex classifiers, and DummyRegressor as a simple baseline to compare against other regressors. The scikit-learn cross-validation documentation handles procedures for checking evaluation performance over data splitting, and Raschka's review literature summarizes that model-evaluation and model-selection procedures are important. The minimum generalization that this Section takes from here is roughly this: score differences become readable only when a simple standard and a candidate model are placed on top of comparable conditions.

The theory needed here can be grouped into three broad points.

Stated very briefly, these three points mean the following.

• a baseline can be read as a simple comparison target that uses the input very little
• only with a baseline does the experiment become one that asks did the input actually help
• only when the baseline and candidate model are placed under comparable evaluation conditions does the score gap become easier to interpret

That means this Section reads the baseline not through specific library syntax, but through the interpretation principle that to talk about improvement, readers must first fix a simple comparison target and then read the score gap under comparable conditions.

What Does A Baseline Model Do?¶

The scikit-learn DummyClassifier documentation explains a classifier that ignores input features and makes predictions as a simple baseline to compare against other more complex classifiers. The DummyRegressor documentation likewise explains a regressor that predicts by simple rules such as the mean or median as a simple baseline.

If this explanation is summarized as briefly as possible, it becomes the following.

A baseline model is the simplest comparison standard that can be built even without deeply understanding the input.

In other words, a baseline is not a finished model meant to solve a real problem well. It is the minimum standard that says if the model is not better than this, there was no point in using a complex model.

Read a little more theoretically, the baseline functions as a minimum comparison target. Even if complex structure is added and much tuning is done, if it is not meaningfully better than the baseline, that experiment may have increased only complexity rather than improvement.

What Kind Of Illusion Appears Without A Baseline?¶

Without a baseline, a high number can look like a good model right away. But in reality it may not be.

For example, in data where only 10% of customers churn, if the model predicts no one churns for everyone, accuracy can still come out to 90%. In this case, the accuracy alone looks good, but the model actually catches none of the important minority class.

In other words, without a baseline, illusions such as the following can appear.

So a baseline is not a device for lowering scores, but a device that makes scores readable.

This book distinguishes baseline model and baseline reference. The former means the simplest prediction standard, and the latter means the comparison frame that interprets recent results by placing them alongside the usual standard. If such a comparison frame is built separately, the scores of rule-based warnings or learned models seen later can be read more clearly as to how meaningful a change they actually represent.

However, the difference between a recent interval and a baseline is not a device that automatically confirms the cause of that difference. It is only a comparison signal that narrows explanation candidates.

That means this book reads the word baseline together at two levels.

Even when reading this table, what matters is the comparison frame more than one absolute value. And for this kind of comparison to be possible, multiple time-point raw series first have to be changed into one-action summary rows so that the same units can be placed side by side.

That is why the baseline reference in this Section should be read not as a separate technique competing with the baseline model, but as an expression that extends the reading principle of setting the comparison standard first into the scene of interpreting operational data.

If this difference is drawn in the simplest way, it becomes the following.

flowchart TD
  subgraph M2["read the score with baseline"]
    direction LR
    C["baseline 0.90"]
    D["model 0.91"]
    E["small gain<br/>check if it is useful"]
  end

  subgraph M1["read the score without baseline"]
    direction LR
    A["score 0.91"]
    B["looks good"]
  end

  E --> A
  A --> B
  C --> D --> E

The core of this diagram is that even the same number is read in a completely different way depending on whether a baseline exists.

The following table is especially important.

In other words, baseline comparison is not adding one more score table. It is an interpretation device that asks is this improvement really the result of reading the structure better.

That is why when reading a baseline, readers should not write only the number, but also leave the following two things immediately together.

• what representative error scene the baseline is still missing
• whether the failure reduced by the candidate model is a failure that is actually important in operations

These questions are the final stage of baseline methodology. The role of the baseline does not end at writing one line of numbers. It goes as far as showing what kind of failure that number still leaves behind.

Where Are Representative Baseline Methods Continued?¶

If this Section first established the need for the baseline and the principle of interpretation, then the natural next question is so what baseline should be built, and how. That question is continued in P4-8.3 supplementary learning: how to first set a baseline by problem type.

That next Section in particular groups the following into actual cases and exercises.

Why Must The Baseline Come Before Tuning?¶

People often proceed like this.

1. Choose a complex model.
2. Change many parameters.
3. If the score rises a little, treat it as success.

But without a baseline, there is no way to know whether that improvement is actually meaningful.

flowchart TB
  A["baseline"]
  B["candidate model"]
  C["compare before tuning"]
  D["tuning after comparison"]
  E["complex model first"]
  F["tuning without baseline"]
  G["score change, but unclear meaning"]

  A --> B --> C --> D
  E --> F --> G

This diagram shows together the correct order of setting the baseline first, comparing the candidate model, and only then entering tuning, and the problem that if a complex model is tuned immediately without a baseline, the meaning of the score change becomes difficult to interpret.

In practice, this difference in order quickly becomes a difference in cost. If a candidate that cannot even beat the baseline is tuned for a long time, experiment time and computational cost may be spent without leaving any improvement that can be explained.

Detailed Learning Content¶

How Should DummyClassifier And DummyRegressor Be Understood?¶

The dummy-family models of scikit-learn are especially useful educationally.

The value of these tools is not for actual service use. It lies in quickly showing what a real model must at least beat.

In other words, a baseline is part of model selection and at the same time part of reading evaluation.

Cases And Examples¶

Case 1. Fraud Detection That Looks High In Accuracy But Actually Catches Nothing¶

A payment-service team is building a classification model that catches fraudulent transactions. The criteria people first used were signals such as repeated payment in a short time, a region different from usual, and payment at a strange hour.

Before building the model, the team first sets up a very simple standard that predicts every transaction as normal. If fraudulent transactions are extremely rare, even this baseline can produce high accuracy. So without a baseline, it is easy to mistakenly think that a complex model improved performance just by looking at its accuracy.

In this scene, the baseline becomes the floor line for checking did the complex model really improve in a meaningful way. Even if the actual model raised accuracy a little, it may not be a big improvement from the operational point of view if fraud recall is still low. By contrast, if recall and F1 clearly improved over the baseline, only then can the reader say that the complex model handles the minority-class problem better.

The confirmable result appears when the baseline and actual model are compared side by side on the same metrics. When not only accuracy but also recall and F1 are placed together, it becomes clear why the baseline is not a low-performance model, but a reference line for interpreting scores.

The example below reduces this illusion with an actual DummyClassifier and DecisionTreeClassifier. The data are created as a classification problem with a small positive class, like fraud transactions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

from sklearn.datasets import make_classification
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, recall_score
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

X, y = make_classification(
    n_samples=260,
    n_features=6,
    n_informative=2,
    n_redundant=0,
    weights=[0.82, 0.18],
    class_sep=0.9,
    random_state=21,
)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.35, random_state=4, stratify=y
)

models = {
    "dummy_most_frequent": DummyClassifier(strategy="most_frequent"),
    "decision_tree": DecisionTreeClassifier(max_depth=3, random_state=0),
}

for name, model in models.items():
    model.fit(X_train, y_train)
    predicted = model.predict(X_test)
    tn, fp, fn, tp = confusion_matrix(y_test, predicted, labels=[0, 1]).ravel()
    print(name)
    print(
        " accuracy=", round(accuracy_score(y_test, predicted), 3),
        "positive_recall=", round(recall_score(y_test, predicted, zero_division=0), 3),
    )
    print(" confusion=", {"tn": int(tn), "fp": int(fp), "fn": int(fn), "tp": int(tp)})

An example output is as follows.

1
2
3
4
5
6

dummy_most_frequent
 accuracy= 0.824 positive_recall= 0.0
 confusion= {'tn': 75, 'fp': 0, 'fn': 16, 'tp': 0}
decision_tree
 accuracy= 0.956 positive_recall= 0.75
 confusion= {'tn': 75, 'fp': 0, 'fn': 4, 'tp': 12}

dummy_most_frequent also looks high in accuracy at 0.824. But positive-class recall is 0.0, and it missed all 16 actual positives. A baseline model is therefore not decoration for showing a failed model. It is a comparison line that first reveals the illusion created when only accuracy is read.

flowchart TD
  A["fraud dataset"]
  B["always predict normal"]
  C["accuracy looks high"]
  D["fraud recall stays near zero"]
  E["train a richer model"]
  F["compare accuracy, recall, and F1"]
  G["decide whether the gain is operationally meaningful"]

  A --> B --> C --> D
  A --> E --> F --> G

Checklist¶

• What is the simplest baseline for the current problem?
• Are the baseline score and the actual model score being compared on the same metric?
• Are you distinguishing whether the model is better than the baseline, or only more complex?
• Have you checked easy traps such as class imbalance or mean-only prediction through the baseline?
• Before tuning, did you first read the difference between the baseline and candidate model?
• Can you explain that a baseline is the comparison standard set before a complex model, and that even a high score is hard to interpret without comparing it to the baseline?
• Can you explain that a baseline is not one more score to inspect, but a standard that leaves together the starting-point score, representative error cases, and key-metric differences?

Sources And References¶

• scikit-learn developers, DummyClassifier, scikit-learn API Reference, accessed 2026-07-09.
• scikit-learn developers, DummyRegressor, scikit-learn API Reference, accessed 2026-07-09.
• scikit-learn developers, Cross-validation: evaluating estimator performance, scikit-learn User Guide, accessed 2026-07-09.
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, accessed 2026-07-09.
• Sebastian Raschka, Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning, arXiv, 2018, accessed 2026-07-09.