Class 10 · Unit 3: Evaluating models · Lesson 3.3 · 40 min
The confusion matrix, from scratch
Wrong ten times can cost a farmer ₹3,000 — or ₹20,000. It depends which wrong.
Today you will: Build a confusion matrix from raw predictions · Name TP, FP, TN and FN · Put a price on each kind of mistake
Story
Hiba: “Eighty-three per cent. So it's wrong ten times in sixty. Wrong how?” ('s uncle.)
Nobody at the table can answer.
Uncle: “Because there are two ways to be wrong. It can call a sick leaf healthy. Then I don't spray, and the scab spreads through the whole row. Or it can call a healthy leaf sick. Then I spray a tree that didn't need it. One of those costs me a bag of spray. The other costs me a season.”
He puts the tablet down.
Hiba: “Don't tell me how often it's wrong. Tell me which wrong.”
Watch
Four boxes that tell the truth — — scan code 10.3.3 in the printed book.
Warm up your fingers
Skill: a count with two conditions (5 min)
-
On typing.com: the symbols lessons.
-
Type three times, eyes on the screen:
=COUNTIFS(C2:C61;"scab";D2:D61;"healthy") -
Goal this term: 38–44 WPM at 95%.
On the laptop
You need: LibreOffice Calc · datasets/leaf-predictions.csv
Mission 1: Build the matrix (12 min, in pairs)
- ☐1
Open
leaf-predictions.csv. Positive class: scab. - ☐2
Somewhere empty, draw this table:
Predicted scab Predicted healthy Actually scab TP FN Actually healthy FP TN - ☐3
Predict: which of the four cells will be biggest? Which smallest? Write both in the cell above the table.
- ☐4
Fill each cell with
COUNTIFS. TP is=COUNTIFS(C2:C61;"scab";D2:D61;"scab"). Work out the other three. - ☐5
Check: all four add to 60. The top row adds to how many leaves were really scab.
Mission 2: Accuracy from the matrix (5 min)
- ☐6
Work out (TP + TN) ÷ 60. It should match Lesson 3.2.
Mission 3: The price of each mistake (8 min)
- ☐7
Hiba's uncle says a miss (FN) costs him about ₹2,000 in spread disease, and a false alarm (FP) about ₹300 of spray. Work out the cost of the model's mistakes.
- ☐8
Build the matrix for Bilal's lazy model from Lesson 3.2, and cost its mistakes the same way.
- ☐9
In the cell under both, finish: "The model with more right answers is … ; the model that costs less is … "
Finished early?
Find the "break-even" price: how cheap would a miss have to be before the lazy model cost less than the real one?
No laptop today?
Twelve leaf cards go round the class: each shows actual and predicted. Four corners of the room are TP, FN, FP and TN. Everyone walks to their corner and the corners count themselves. Then cost the mistakes on the board.
Now you know
- Choose the positive class first the thing the model is looking for. Here, scab.
- The four cells. TP and TN are right. FN is a miss. FP is a false alarm.
- Actual down the side, predicted across the top. Each row adds up to how many there really were.
- Accuracy from the matrix = (TP + TN) ÷ everything.
- Mistakes have different prices. A miss and a false alarm are both "wrong", and they can cost completely different amounts. The matrix shows which you are making.
Debate it
The ₹2,000 and ₹300 were the uncle's guesses.
- Who else pays when a miss lets scab spread? (Think about the orchard next door.)
- For a model that screens people for a disease, what would a false negative cost — and could anyone put a rupee figure on it?
Check yourself — practice, not a test
1. Match each case to its cell (positive = scab):
Really scab, predicted scab · Really scab, predicted healthy · Really healthy, predicted scab · Really healthy, predicted healthy
Match with: True Positive · False Negative · True Negative · False Positive2. A false alarm is a:
- ○ True Positive
- ○ False Positive
- ○ False Negative
- ○ True Negative
3. Accuracy from a confusion matrix = (TP + ______) ÷ all predictions.
4. In CBSE's confusion matrix, the actual values run down the side and the predicted values across the top.
- ○ True
- ○ False
5. TP = 11, FN = 4, FP = 6, TN = 39. How many leaves were really scab?
- ○ 11
- ○ 15
- ○ 17
- ○ 50
Remember
Don't ask how often it's wrong. Ask which wrong.