P4-8.3 Supplementary Learning: How To First Set A Baseline By Problem Type¶

Section ID: P4-8.3 Version: v2026.07.20

If P4-8.2 showed why a baseline is needed, then the next question immediately appears.

So how is a baseline actually set?

This supplementary learning answers that question. Its goal is not to memorize many baseline names, but to enable the reader to look at the problem type and directly choose the simplest standard that still carries comparison meaning.

Scope Of This Supplementary Learning¶

This Section handles how to first set representative baselines in classification, regression, and time-series problems.

• What should be fixed first before setting a baseline?
• What baseline can be brought to mind first depending on problem type?
• After setting a baseline, with what cases and exercises should comparison begin?
• After seeing the baseline score, what should be checked next?

This Section first closes how to set the simplest baseline that still carries comparison meaning by problem type. Cross-validation, model-comparison procedures, and more complex tuning methods continue after P4-9.

Goals Of This Supplementary Learning¶

• You can distinguish representative baseline candidates in classification, regression, and time series.
• You can explain the baseline-setting procedure in the order fix the problem type -> choose the simplest rule -> measure with the same metric -> inspect errors -> decide the next comparison.
• You can explain why cases and exercises should be attached immediately after the baseline explanation.
• You can say to what next question the reader should proceed after seeing the baseline score.

First Fix The Baseline-Setting Procedure¶

It is safer to set a baseline not as a temporary rule chosen by intuition, but as a procedure for choosing the simplest standard in the current problem that still carries comparison meaning.

The shortest procedure can be fixed in the following order.

1. Fix the problem type first.
2. Choose the simplest rule that uses features very little.
3. Measure the baseline on the same metric as the candidate model.
4. Look not only at the score, but also at representative error scenes.
5. Only after interpreting how much better it is than the baseline should the reader move to tuning or changing candidates.

If this order is grouped again in a table, it becomes the following.

In other words, the core of baseline methodology is not make the easiest rule, but place the easiest rule inside the same comparison frame.

Representative Baseline Map By Problem Type¶

Representative baselines begin from different starting points depending on problem type.

This table is not an answer list of baselines, but a map that quickly organizes what should be set as the comparison line first. On the time-series side, methods such as naive and seasonal naive are directly explained in public teaching material such as Hyndman et al. as representative simple forecasting methods.

How Is A Baseline Set In Classification?¶

In classification, readers can first think of baselines such as the following.

Consider a case where churn is rare in customer-churn prediction. A baseline that says always non-churn can produce high accuracy. But churn recall can become 0. That is why the first role of a classification baseline is not to answer is the score high, but to reveal first what illusion is being created.

Cases And Examples¶

Case 1. Why A Majority-Class Baseline Is Set First In Customer-Churn Prediction¶

A subscription-service team is trying to predict customer churn next month. Suppose the data contain 90% non-churn customers and 10% churn customers.

If the baseline predicts always non-churn, accuracy can become 90%. But what matters is that it catches no churn customers at all. So if the reader looks only at the actual model's accuracy of 91% without the baseline, the reader ends up saying it improved by 1 percentage point. But once the baseline is placed beside it, the first question becomes is it still catching almost none of the important customers.

flowchart TD
  A["churn data"]
  B["baseline: always stay"]
  C["accuracy looks high"]
  D["churn recall is 0"]
  E["candidate model"]
  F["compare accuracy, recall, F1"]
  G["check missed churn cases"]

  A --> B --> C --> D
  A --> E --> F --> G

The key point of this case is that the baseline matters not because it is a low model, but because it is the standard that immediately reveals the accuracy illusion.

Small Example: Reading A Classification Baseline Numerically¶

If the baseline and actual model are placed side by side on the same data as follows, the role of the baseline becomes clearer.

The questions for reading this table are simple.

• Is the difference in accuracy actually meaningful?
• How much did recall and F1 change from the baseline?
• What kinds of churn customers are still being missed?

How Is A Baseline Set In Regression?¶

In regression, the following baselines can be placed first.

The core of a regression baseline is to first create the starting point that says this much error appears even when there is no input.

Case 2. Why A Mean Baseline Comes First In Delivery-Time Prediction¶

A logistics team wants to predict delivery time. Before building a model that uses region, distance, volume, and weather information, it first places a baseline that predicts only the average delivery time.

The purpose of this baseline is not that average prediction is excellent. It is to check how much the model using input features actually reduces error compared with the average standard.

For example, if the MAE of the mean baseline is 18 minutes and the MAE of the candidate model is 15 minutes, the next question becomes is a reduction of 3 minutes operationally meaningful. To answer that, the reader should look not only at the overall average but also at in what order groups the error actually decreased.

Small Example: Reading A Regression Baseline Numerically¶

When reading this table, it is safer to also write down the following together.

• Which order groups saw large errors reduced?
• Did it improve not only on average, but also in important regions?
• Does the reduced error over the baseline lead to operational cost reduction?

How Is A Baseline Set In Time Series?¶

In time-series problems, rather than erasing time order completely, it is often more natural to set a simple standard that reflects time structure at least minimally.

Case 3. Why Seasonal Naive Comes First In Daily Visitor Forecasting¶

An e-commerce team wants to forecast daily visitors. If the pattern difference between Mondays and weekends is large, it is more natural to use the value from the same weekday last week as the next standard than to use a simple mean.

For example, if seasonal naive, which uses the value from the same weekday last week, produces lower error than a naive baseline that simply uses yesterday's visitor count, that means the weekday cycle must be part of the minimum standard in that time series.

Small Example: Comparing Time-Series Baselines¶

In this scene, the baseline itself may not be only one. In time series, it is more natural to first compare whether the previous-value standard or the cycle standard is the better floor line, and then place a complex model above that.

Practice And Examples¶

Comparing A Classification Baseline And Candidate Model Through A Python Example¶

The example below is a tiny exercise that compares scikit-learn's DummyClassifier and a simple classification model.

Problem situation:

• to see whether a new model is really helpful, it must be placed beside a minimum standard and compared

Input:

• imbalanced classification data made with make_classification
• DummyClassifier
• LogisticRegression

Expected output:

• accuracy, recall, and F1 of the baseline and actual model

Concepts to check:

• the baseline is the comparison standard that a complex model must at least beat
• in imbalanced data, not only accuracy but also recall and F1 should be viewed together

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

# This example sets baselines by problem type and compares candidate model performance against them.
from sklearn.datasets import make_classification
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, recall_score, f1_score
from sklearn.model_selection import train_test_split

X, y = make_classification(
    n_samples=400,
    n_features=6,
    n_informative=3,
    n_redundant=0,
    weights=[0.9, 0.1],
    random_state=42,
)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, stratify=y, random_state=42
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)
baseline_pred = baseline.predict(X_test)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
model_pred = model.predict(X_test)

print("baseline accuracy :", round(accuracy_score(y_test, baseline_pred), 3))
print("baseline recall   :", round(recall_score(y_test, baseline_pred), 3))
print("baseline f1       :", round(f1_score(y_test, baseline_pred), 3))
print()
print("model accuracy    :", round(accuracy_score(y_test, model_pred), 3))
print("model recall      :", round(recall_score(y_test, model_pred), 3))
print("model f1          :", round(f1_score(y_test, model_pred), 3))

An example of the execution result is as follows.

1
2
3
4
5
6
7

baseline accuracy : 0.9
baseline recall   : 0.0
baseline f1       : 0.0

model accuracy    : 0.942
model recall      : 0.5
model f1          : 0.632

What must be read in this example is not one number.

• accuracy can already be high even for the baseline
• recall and F1 can show better whether important minority classes are actually being caught
• that is why baseline comparison makes the reader look together at on what metric improvement happened and what kind of error remains

Actually Comparing Regression Baselines Through A Python Example¶

The example below uses a very small target-value array to compare the mean baseline and median baseline directly.

Problem situation:

• in delivery-time prediction, the reader wants to see whether the mean baseline or median baseline is the more natural starting point

Input:

• simple training target values and evaluation target values
• mean baseline, median baseline

Expected output:

• the predicted values and MAE of the mean baseline and the median baseline

Concept to check:

• even for a baseline, the more natural starting point can change depending on the actual data scene

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

# This example sets baselines by problem type and compares candidate model performance against them.
y_train = [32, 35, 31, 120, 33]
y_test = [34, 36, 30, 90]

mean_value = sum(y_train) / len(y_train)
median_value = sorted(y_train)[len(y_train) // 2]

mean_pred = [mean_value] * len(y_test)
median_pred = [median_value] * len(y_test)

def mae(y_true, y_pred):
    return sum(abs(a - b) for a, b in zip(y_true, y_pred)) / len(y_true)

print("mean baseline")
print("  prediction :", [round(v, 1) for v in mean_pred])
print("  MAE        :", round(mae(y_test, mean_pred), 2))
print()
print("median baseline")
print("  prediction :", [round(v, 1) for v in median_pred])
print("  MAE        :", round(mae(y_test, median_pred), 2))

An example of the execution result is as follows.

1
2
3
4
5
6
7

mean baseline
  prediction : [50.2, 50.2, 50.2, 50.2]
  MAE        : 22.6

median baseline
  prediction : [33, 33, 33, 33]
  MAE        : 16.0

This example shows why both the mean baseline and the median baseline should be brought to mind. If the training target values include one very large value, 120, the mean gets dragged strongly, while the median stays more stable. In a scene like this, it is safer to first check numerically whether the mean is the default baseline or the median is the more natural baseline.

Checklist¶

• Have you written down at least one representative baseline candidate that fits the current problem type?
• Are you comparing the baseline and the candidate model with the same metric?
• Are you looking together not only at score difference, but also at representative error scenes?
• Are you avoiding dragging a candidate that cannot beat the baseline into long tuning?
• Can you explain that a baseline is not the model that loses first, but the standard that must be compared first?
• Can you explain that representative baseline methods must be set differently depending on problem type?
• Can you explain that it is safer to move to tuning only after confirming that a candidate better than the baseline exists?

Sources And References¶

• scikit-learn developers, DummyClassifier, scikit-learn API Reference, accessed 2026-07-09.
• scikit-learn developers, DummyRegressor, scikit-learn API Reference, accessed 2026-07-09.
• scikit-learn developers, Cross-validation: evaluating estimator performance, scikit-learn User Guide, accessed 2026-07-09.
• Rob J Hyndman, George Athanasopoulos, Forecasting: Principles and Practice (3rd ed), 5.2 Some simple forecasting methods, accessed 2026-07-09.
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, accessed 2026-07-09.
• Sebastian Raschka, Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning, arXiv, 2018, accessed 2026-07-09.