P4-6.2 Evaluation Criteria By Problem Type¶

Section ID: P4-6.2 Version: v2026.07.20

P4-6.1 explained that an evaluation metric is not just a scoreboard. It is a criterion that reveals what we choose to treat as important. Now the next question follows. If the problem changes, why do the metrics we inspect also change?

The answer is simple. The model's output changes, and the judgment connected to that output changes as well. Classification is the problem of choosing a category, regression is the problem of predicting a number, and clustering is the problem of grouping similar things together. So the meaning of did well cannot stay the same.

Scope Of This Section¶

This Section is an introductory organization of why evaluation criteria change across classification, regression, and clustering. It connects the questions that should come to mind for each problem type with representative metrics. Introductory explanations of ROC curve, PR curve, log loss, calibration, and silhouette coefficient are collected separately in the P4-6.4 supplementary Section, and clustering-quality interpretation is connected again in P4-17.1 and P4-17.2.

In the core text, the first scope is the following: which kinds of errors hurt more in classification, how far off is the model on average in regression, and how should structure be read in clustering when there may be no answer label. More delicate readings such as calibration curves and Brier score are first collected in P4-6.4, while actual threshold adjustment and the connection to service policy return in P4-15.3.

This Section answers the following questions.

• What differs in evaluation across classification, regression, and clustering?
• Why does classification not end with accuracy alone?
• Why does regression inspect how far off was it rather than just right or wrong?
• Why does it matter in clustering that an answer label may not exist?
• Why does it become easier to read later algorithm chapters once the problem type is separated first?

Goals Of This Section¶

• You can explain that the evaluation question changes by problem type.
• You can explain why representative metrics for classification and regression are different.
• You can explain that clustering often becomes more delicate because there is frequently no answer label.
• You can prepare for what evaluation questions follow later when learning algorithms such as linear regression, logistic regression, k-NN, and decision trees.

Learning Background¶

First Separate The Problem Type¶

The scikit-learn documentation explains evaluation functions by separating them according to the goal of the problem. The fact that classification metrics, regression metrics, and clustering metrics are separate is itself an important hint. It means that different questions are hidden under the same word performance.

flowchart TB
  A["problem type"]
  B["classification<br/>choose a class"]
  C["regression<br/>predict a number"]
  D["clustering<br/>group similar samples"]

  E["read mistakes by type<br/>accuracy / precision / recall / F1"]
  F["read error size<br/>MAE / MSE / RMSE / R2"]
  G["read group quality<br/>label agreement or silhouette"]

  A --> B
  A --> C
  A --> D
  B --> E
  C --> F
  D --> G

This diagram shows that once the problem type is chosen first, the evaluation question that should be read next also changes naturally. What should be read first here is not the metric name, but the distinction is the current output a category, a number, or a grouping?

The criterion to fix first in this Section is the following table.

The point readers confuse most often here is that when the shape of the output changes, the meaning of a good result also changes.

The evaluation order by problem type can be fixed briefly like this.

So classification is closer to a problem of attaching a name label, regression is closer to a problem of matching a numeric scale, and clustering is closer to a problem of finding groups that were not named in advance by people.

Main Learning Content¶

In Classification, Read The Types Of Errors First¶

In classification, it is often not enough to read the prediction result only as correct or wrong. As already seen in P4-6.1, the mistake of missing an actual positive case and the mistake of calling a negative case positive for no good reason can have different costs.

The scikit-learn documentation also separates classification metrics into items such as accuracy, F1, confusion matrix, ROC AUC, and precision-recall curve. That means classification performance does not end in one number.

Classification becomes clearer when read through the following order.

1. What counts as positive, and what counts as negative?
2. Does it hurt more to miss a case, or to raise a false alarm?
3. So should recall be the first metric to inspect, or precision?
4. Is there a need to inspect F1 score or another metric together to read the overall balance?

Even within the same classification problem, the questions can differ like this.

So in classification, the first thing to read is the kind of error.

Reading Classification More Concretely¶

Classification looks simple on the surface because it only chooses one option among categories. But in practice, two layers are mixed together.

1. Which class does the model consider more likely?
2. How is that likelihood turned into an actual decision in service?

For example, a mail filter may produce an internal score such as spam probability 0.82. But users do not see a probability. They experience decisions such as move to spam, send to review, or keep in inbox. So classification evaluation goes beyond a simple label table and also ends up reading the error structure created by that decision.

The following distinctions are especially important when reading classification.

For example, in medical screening, classifying too many cases as positive may be a smaller problem than missing an actual patient. By contrast, in an automatic approval system, letting a risky target pass incorrectly may be the bigger problem. So classification is both the task of choosing a category and the task of managing the kinds of errors.

Social And Work Examples Of Classification¶

Classification is one of the most widely used problem types in reality. The reason is that many institutions and services eventually move through category judgments such as pass/hold, normal/anomaly, allow/block, and approve/reject.

Classification is also very common in work systems.

These examples show that classification is more accurately understood not merely as technology for attaching labels, but as technology that changes the next action taken by people and organizations.

In Regression, Read The Size Of Error First¶

Regression is the problem of predicting numbers. It is enough to imagine situations such as house price, delivery time, power usage, sales, or temperature. Here, what matters more is not simply wrong, but how wrong.

The scikit-learn documentation separately explains regression metrics such as mean absolute error, mean squared error, and R² score. That structure itself means that in regression, the size of error and the way error is interpreted are central.

Here the distinction is organized like this.

The important point here is that regression metrics are connected to the interpretation of numeric difference.

• In delivery-time prediction, a 1-minute error and a 30-minute error are not the same mistake.
• In power-demand prediction, a large error can shake the whole operating plan.
• In house-price prediction, the average size of the difference may be more intuitive.

So in regression, the key is not whether error exists, but the size and cost of the error.

Reading Regression More Concretely¶

Because regression deals with numbers, many readers easily misunderstand it as a problem of matching one exact correct answer. But in practice, the key is how close a number the model gives.

For example, if the true house price should be 500 million won, then 501 million won and 700 million won are both wrong, but they are not the same kind of wrong. Regression metrics let that difference be read.

The following questions come attached when reading regression.

The sense of baseline is also important here. In regression, a good model is not automatically a model that hit the exact value. At the very least, it should be possible to say it is better than a model that mindlessly says the average. R² helps this comparison perspective, but a high number does not always mean the model is enough in practice.

Regression is also sensitive to the unit.

• MAE of 1 million won in house-price prediction
• MAE of 1 million won in delivery-time prediction

Even if the number looks the same, the meaning is completely different. So in regression, the number itself must be read together with the unit in which that error exists.

Social And Work Examples Of Regression¶

Regression can feel less familiar than classification at first, but it appears extremely often in actual work. The reason is that organizations constantly handle numbers such as how much, until when, how many, and what cost when making decisions.

Regression also appears often in work analysis.

When reading regression examples, it is not enough to think only about predicting a number. You also have to ask how much does the error in that number shake planning and operations? Regression evaluation is both mathematical error and operational error.

In Clustering, There May Be No Answer At All¶

Clustering may feel stranger than classification or regression. The reason is simple. Clustering usually begins from a state where there is no answer label.

The scikit-learn documentation explains clustering performance evaluation by separating the case where answer labels are known from the case where they are not. It explicitly says that this is not as simple as precision and recall in supervised learning.

This difference is extremely important.

flowchart TD
  A["clustering result"]
  B["ground-truth labels available?"]
  C["yes<br/>compare two labelings"]
  D["no<br/>inspect cluster structure itself"]
  E["ARI / FMI / homogeneity-completeness"]
  F["silhouette or other internal criteria"]

  A --> B
  B --> C
  B --> D
  C --> E
  D --> F

The key point of this diagram is that clustering evaluation splits into two branches from the beginning. If human answer labels exist, the question becomes how similar is this to the human grouping? If no such labels exist, the question becomes how plausible is the grouping itself?

So in clustering, the fact that it is evaluation without an answer is itself a core learning point.

Reading Clustering More Concretely¶

Clustering can feel the most abstract. Classification has an answer category, and regression has a number to match. But in clustering, from the beginning, a person must interpret again what kind of grouping counts as a good grouping?

So when reading clustering, what the model did has to be split into two layers.

1. What grouping structure did it create inside the data?
2. Does that grouping also match distinctions that a human can understand?

For example, when customer data are clustered, the model may output only IDs such as cluster 1, cluster 2, and cluster 3. But those numbers themselves do not automatically mean loyal customers, customers at risk of churn, or occasional visitors. A human has to read and interpret that meaning again.

The following distinctions matter.

So clustering evaluation must think about both does the shape look plausible? and is it interpretable? In that sense, clustering brings in more interpretive work of looking at structure, comparing, and naming than simple score reading.

Social And Work Examples Of Clustering¶

Clustering is closer to finding hidden structure than to matching a known answer. That is why it appears often in early analysis, policy planning, and service operation.

In work settings, clustering usually leads to the following questions.

So clustering should be understood less as a problem where the model gives the final answer and more as a problem where the model reveals structure first so that people can interpret further.

Detailed Learning Content¶

Misunderstanding Appears If The Same Kind Of Number Is Expected Across Problem Types¶

It is easy to think like this.

• classification also has a score
• regression also has a score
• clustering also has a score
• so in the end, doesn't a bigger number always mean better?

This reading misses an important difference.

So before looking at a metric, the first check should be what is this model predicting?

In evaluation by problem type, the first question is fixed by what is this model predicting right now?

Cases And Examples¶

Case 1. Even With The Same Customer Data, The Evaluation Criterion Changes When The Question Changes¶

A service team is reviewing several tasks on the same customer dataset. People first split the work questions, such as should we find customers likely to churn, should we predict next month's purchase amount, and should we divide customers into similar groups.

The problem begins when the team expects the same kind of performance number simply because the dataset is the same. Churn prediction is a classification problem, so misses and false alarms must be examined. Purchase-amount prediction is a regression problem, so the size of error must be inspected. Customer segmentation is clustering, so the question becomes whether the grouping is interpretable even without answer labels. If everything is bundled under the same word score, the judgment criterion becomes blurry.

Here, separating the problem type becomes the starting point that changes the evaluation criterion. If it is classification, the first questions are around precision, recall, and F1. If it is regression, the size of error is read through metrics such as MAE or RMSE. If it is clustering, silhouette and human interpretability must be read together.

The checkable result is also read differently. Even with the same customer data, classification requires checking the number of missed positive cases, regression requires checking average error size, and clustering requires checking compactness within clusters and separation between clusters. In other words, the first thing that must change is not the data, but the question.

It becomes even clearer when the same customer table is drawn as a diagram that branches into different evaluation questions.

flowchart TD
  A["same customer table"]
  B["classification question<br/>who may churn?"]
  C["regression question<br/>how much will they spend?"]
  D["clustering question<br/>which groups appear?"]
  E["read FP / FN and F1"]
  F["read error size<br/>MAE / RMSE"]
  G["read group structure<br/>silhouette / interpretation"]

  A --> B
  A --> C
  A --> D
  B --> E
  C --> F
  D --> G

Cases And Examples¶

Reading It Again Through Very Short Scene Comparisons¶

Even with the same data task, once the problem type changes, the reader's question changes completely.

This table shows that the problem sentence must be rewritten before the algorithm is discussed. Even with the same dataset, when the question changes, the problem type changes, and then the evaluation criterion changes with it.

Practice And Examples¶

Trying Classification Through A Python Example¶

In classification, precision and recall can change even with a small shift in threshold. The following example shows that even with the same score, the result changes depending on from what score onward should the case be treated as positive.

The example below uses the true labels y_true, predicted scores scores, and several threshold values. In the result, inspect the prediction result by threshold, TP/TN/FP/FN, accuracy, precision, and recall together.

The key to check is that classification evaluation must be read as a final judgment after the threshold, not only as model scores. The balance between precision and recall can change depending on the threshold.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# This example computes and compares different evaluation criteria for classification, regression, and clustering problems.
y_true = [1, 1, 1, 0, 0, 0, 1, 0]
scores = [0.95, 0.80, 0.55, 0.70, 0.40, 0.20, 0.45, 0.60]

def classification_metrics(y_true, scores, threshold):
    y_pred = [1 if s >= threshold else 0 for s in scores]

    tp = sum(1 for yt, yp in zip(y_true, y_pred) if yt == 1 and yp == 1)
    tn = sum(1 for yt, yp in zip(y_true, y_pred) if yt == 0 and yp == 0)
    fp = sum(1 for yt, yp in zip(y_true, y_pred) if yt == 0 and yp == 1)
    fn = sum(1 for yt, yp in zip(y_true, y_pred) if yt == 1 and yp == 0)

    accuracy = (tp + tn) / len(y_true)
    precision = tp / (tp + fp) if (tp + fp) else 0
    recall = tp / (tp + fn) if (tp + fn) else 0

    return {
        "threshold": threshold,
        "pred": y_pred,
        "tp": tp,
        "tn": tn,
        "fp": fp,
        "fn": fn,
        "accuracy": round(accuracy, 3),
        "precision": round(precision, 3),
        "recall": round(recall, 3),
    }

for threshold in [0.4, 0.6, 0.8]:
    result = classification_metrics(y_true, scores, threshold)
    print("threshold =", result["threshold"])
    print("  pred      =", result["pred"])
    print("  TP TN FP FN =", result["tp"], result["tn"], result["fp"], result["fn"])
    print("  accuracy  =", result["accuracy"])
    print("  precision =", result["precision"])
    print("  recall    =", result["recall"])

The output is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

threshold = 0.4
  pred      = [1, 1, 1, 1, 1, 0, 1, 1]
  TP TN FP FN = 4 1 3 0
  accuracy  = 0.625
  precision = 0.571
  recall    = 1.0
threshold = 0.6
  pred      = [1, 1, 0, 1, 0, 0, 0, 1]
  TP TN FP FN = 2 2 2 2
  accuracy  = 0.5
  precision = 0.5
  recall    = 0.5
threshold = 0.8
  pred      = [1, 1, 0, 0, 0, 0, 0, 0]
  TP TN FP FN = 2 4 0 2
  accuracy  = 0.75
  precision = 1.0
  recall    = 0.5

This example can be experimented with like this.

• change the threshold to 0.5, 0.7, and 0.9
• raise or lower one score slightly
• check in which cases recall goes up while precision goes down

So in classification, it is important to read through the flow model score -> threshold -> final judgment.

Trying Regression Through A Python Example¶

In regression, it is important to develop a feel for reading how large the error is in numbers. In particular, you need to see directly how much one large error shakes the metric.

The example below uses actual values y_true and predicted values y_pred to print absolute_errors, squared_errors, MAE, MSE, and RMSE.

The key to check is that regression metrics are criteria for reading how far numbers were off. MAE and MSE/RMSE read the same errors with different emphasis.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# This example computes and compares different evaluation criteria for classification, regression, and clustering problems.
y_true = [10, 12, 9, 15]
y_pred = [11, 10, 8, 18]

absolute_errors = [abs(a - b) for a, b in zip(y_true, y_pred)]
squared_errors = [(a - b) ** 2 for a, b in zip(y_true, y_pred)]

mae = sum(absolute_errors) / len(absolute_errors)
mse = sum(squared_errors) / len(squared_errors)
rmse = mse ** 0.5

print("absolute_errors:", absolute_errors)
print("squared_errors :", squared_errors)
print("mae            :", round(mae, 2))
print("mse            :", round(mse, 2))
print("rmse           :", round(rmse, 2))

The output is as follows.

1
2
3
4
5

absolute_errors: [1, 2, 1, 3]
squared_errors : [1, 4, 1, 9]
mae            : 1.75
mse            : 3.75
rmse           : 1.94

These numbers can be read like this.

• MAE means it was off by 1.75 on average.
• MSE reflects large errors more strongly.
• RMSE helps the reader return MSE to the feel of the original unit.

So in regression, how much was it wrong sits at the center of the metric.

Now insert one large error.

The example below compares a small-error prediction y_pred_small_error with a prediction that includes a large error, y_pred_big_error. In the result, inspect how absolute error, squared error, MAE, MSE, and RMSE change in the two cases.

The key to check is that MSE and RMSE react more sensitively when you want to penalize large failures more strongly. Even in the same regression problem, you need to choose which type of error to weigh more heavily.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# This example computes and compares different evaluation criteria for classification, regression, and clustering problems.
y_true = [10, 12, 9, 15]
y_pred_small_error = [11, 10, 8, 18]
y_pred_big_error = [11, 10, 8, 30]

def regression_metrics(y_true, y_pred):
    absolute_errors = [abs(a - b) for a, b in zip(y_true, y_pred)]
    squared_errors = [(a - b) ** 2 for a, b in zip(y_true, y_pred)]
    mae = sum(absolute_errors) / len(absolute_errors)
    mse = sum(squared_errors) / len(squared_errors)
    rmse = mse ** 0.5
    return absolute_errors, squared_errors, round(mae, 2), round(mse, 2), round(rmse, 2)

for name, pred in [
    ("small_error_case", y_pred_small_error),
    ("big_error_case", y_pred_big_error),
]:
    absolute_errors, squared_errors, mae, mse, rmse = regression_metrics(y_true, pred)
    print(name)
    print("  pred           =", pred)
    print("  absolute_error =", absolute_errors)
    print("  squared_error  =", squared_errors)
    print("  mae            =", mae)
    print("  mse            =", mse)
    print("  rmse           =", rmse)

The output is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

small_error_case
  pred           = [11, 10, 8, 18]
  absolute_error = [1, 2, 1, 3]
  squared_error  = [1, 4, 1, 9]
  mae            = 1.75
  mse            = 3.75
  rmse           = 1.94
big_error_case
  pred           = [11, 10, 8, 30]
  absolute_error = [1, 2, 1, 15]
  squared_error  = [1, 4, 1, 225]
  mae            = 4.75
  mse            = 57.75
  rmse           = 7.6

This example directly raises the following questions.

• When one large error enters, which metric shakes more?
• Do we want to see average error, or do we want to punish large failures more strongly?

So regression metrics are tools that choose not only how wrong was the number but also which kind of wrong should be treated as heavier.

Trying Clustering Through A Python Example¶

Clustering becomes easier to understand when readers try it by hand. The following example is a very simple experiment that groups one-dimensional values by a distance criterion.

The example below uses a list of one-dimensional values points and several gap values to check how the grouping result changes according to the distance criterion.

The key to check is that the number of clusters and the composition of clusters change depending on the distance criterion. Clustering evaluation must be read together with the later step where people interpret the result again.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# This example computes and compares different evaluation criteria for classification, regression, and clustering problems.
points = [1.0, 1.2, 1.4, 4.8, 5.0, 8.5]

def cluster_by_gap(points, gap):
    clusters = [[points[0]]]

    for value in points[1:]:
        if value - clusters[-1][-1] <= gap:
            clusters[-1].append(value)
        else:
            clusters.append([value])

    return clusters

for gap in [0.3, 0.6, 1.5]:
    print("gap =", gap)
    print("  clusters =", cluster_by_gap(points, gap))

The output is as follows.

1
2
3
4
5
6

gap = 0.3
  clusters = [[1.0, 1.2, 1.4], [4.8, 5.0], [8.5]]
gap = 0.6
  clusters = [[1.0, 1.2, 1.4], [4.8, 5.0], [8.5]]
gap = 1.5
  clusters = [[1.0, 1.2, 1.4], [4.8, 5.0], [8.5]]

Now you can change the values a little and check how the clustering criterion changes.

• add 6.1 to points
• change gap to 2.0
• check at what moment the number of clusters suddenly shrinks

For example, if 6.1 is added and gap = 1.5, then [4.8, 5.0, 6.1] may be read as one cluster. So clustering is closer not to a calculation that matches a correct answer, but to an experiment that asks where the criterion of similarity should be placed.

Checklist¶

• Can you explain why classification, regression, and clustering cannot help but give different meanings to a good result?
• Can you explain why in regression the first thing to read is how far off was it rather than simply right-or-wrong?
• Can you explain why clustering evaluation must include interpretability and structure reading together with a score?
• Can you explain that when the problem type changes, the meaning of good performance changes as well?
• Can you explain why classification should read the kind of error first, regression the size and cost of error first, and clustering structure interpretation first?
• Can you explain that before choosing a metric, the model output and the usage scene must be checked first?

Sources And References¶

• scikit-learn developers, Metrics and scoring: quantifying the quality of predictions, scikit-learn User Guide, accessed 2026-07-19. https://scikit-learn.org/stable/modules/model_evaluation.html
• scikit-learn developers, Clustering performance evaluation, scikit-learn User Guide, accessed 2026-07-19. https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation