Class 10 · Unit 3: Evaluating models · Lesson 3.2 · 40 min
Accuracy and error
Bilal built a leaf model in two seconds. It's 75% accurate. It has never caught a single scabbed leaf.
Today you will: Work out error and accuracy for a price · Work out accuracy for classes · Catch a lazy model hiding behind a good number
Story
The exhibition team have tested their leaf app properly this time, on 60 new photos. The new sign says 83% accurate.
Bilal is not impressed. "I can beat that in two seconds."
He writes a model on a scrap of paper. It has one rule: always say healthy.
Hiba checks it on the same 60 photos. Most leaves in spring are healthy, so most of the time he is right.
Hiba: “Seventy-five per cent. Nearly as good as theirs.”
Bilal: “Told you.”
Hiba: “Bilal. Your model has never found a scabbed leaf in its life.”
Watch
How right, how wrong — — scan code 10.3.2 in the printed book.
Warm up your fingers
Skill: a formula with ABS (5 min)
-
On typing.com: the symbols lessons.
-
Type three times, eyes on the screen:
=1-ABS(B2-C2)/B2 -
Goal this term: 38–44 WPM at 95%.
On the laptop
You need: LibreOffice Calc · datasets/mandi-prices.csv · datasets/leaf-predictions.csv
Mission 1: Error for a number (10 min, in pairs)
- ☐1
In
mandi-prices.csv, a model learned from weeks 1–12 only. Its trend is: price = 617.6 + 18.93 × week. In C14, predict week 13:=617.6+18.93*A14. Fill down to week 16. - ☐2
Predict: which of weeks 13–16 will the model get most wrong? Write it in F1.
- ☐3
In D, E and F work out error
=ABS(B14-C14), error rate=D14/B14and accuracy=1-E14. Then the mean accuracy of the four weeks.
Mission 2: Accuracy for classes (7 min)
- ☐4
Open
leaf-predictions.csv: 60 leaves, with what each really was (actual) and what the model said (predicted). - ☐5
Count the correct ones:
=SUMPRODUCT(C2:C61=D2:D61). Divide by 60. That is the model's classification accuracy. What is its error rate?
Mission 3: Bilal's lazy model (6 min)
- ☐6
In E1 type
lazy. Fill E2:E61 with the wordhealthy. - ☐7
Work out the lazy model's accuracy the same way. Then count how many scab leaves it caught:
=COUNTIFS(C2:C61;"scab";E2:E61;"scab"). - ☐8
In H1 finish: "The lazy model is … % accurate and catches … scabbed leaves, because …"
Finished early?
How many healthy leaves would there need to be, out of 60, for Bilal's lazy model to score higher than the real one?
No laptop today?
Do the two traders from the video on the board, step by step. Then your teacher reads 20 leaves (actual and predicted). Tally the correct ones, then score "always healthy" on the same 20.
Now you know
- Error is how far a prediction is from the truth. For a number: |actual − predicted|.
- Error rate and accuracy. Error rate = error ÷ actual. Accuracy = 1 − error rate.
- Classification accuracy = correct predictions ÷ all predictions.
- Aim: maximise accuracy, minimise error. No real model reaches 100%.
- Accuracy can lie on unbalanced data. When one class is much more common, "always guess the common one" scores well and is useless. The next two lessons fix this.
Debate it
Hospitals test for diseases that most people don't have.
- A test that always says "you're fine" would be very accurate. Why would it be the worst test in the hospital?
- What would you want to know about a test, apart from its accuracy?
Check yourself — practice, not a test
1. A model predicts ₹900; the real price is ₹1,000. What is the error?
- ○ ₹900
- ○ ₹100
- ○ 0.9
- ○ 10%
2. Accuracy = 1 − ______ rate.
3. A model that always predicts the most common class can have a high accuracy and still be useless.
- ○ True
- ○ False
4. A model is right on 45 of 60 leaves. Its classification accuracy is:
- ○ 45%
- ○ 60%
- ○ 75%
- ○ 15%
5. A dataset where one class is far more common than the other is called:
- ○ Labelled
- ○ Unbalanced
- ○ Clustered
- ○ Continuous
Remember
Accuracy is the first question about a model, never the last.