Class 10 · Unit project
Unit 4 project: Find the pictures that break it
Build an image classifier. Then spend the rest of the period trying to break it — on purpose, and in the valley's own light.
The task
Companies pay people to break their own systems before anyone else does. It is called red teaming, and it is one of the most useful jobs in AI.
Today your group builds an image classifier — and then becomes its red team. Your job is not to make it look good. Your job is to find exactly where it fails, and prove it with numbers.
Work in your project groups of 3–4. One period. Everything goes in break-it.odt.
Build it (10 min)
- Choose three classes you can photograph easily and safely — three kinds of leaves, three fruits, three kinds of school stationery. No people, no faces.
- In Teachable Machine, train on the ordinary photos your group took before the lesson: at least 15 per class, in normal light, on a plain background.
Break it (18 min)
- Test it under five conditions. For each, use 3 photos per class (9 per condition),
taken before the lesson. Choose at least three from this list, and invent the rest:
- Snow glare — bright white light, as on a winter morning
- Dim light — a winter evening, or a power cut with a phone torch
- Busy background — a market stall, a patterned carpet, a pheran
- Part hidden — half the object covered by a hand or another object
- Odd angle — from directly above, or very close
- Old camera — photos from the oldest phone anyone can find
- For each condition, record actual and predicted, and count how many of 9 it gets right.
- Build the full confusion matrix for the worst condition. Which class does it confuse with which?
Report it (10 min)
Fill the four parts of break-it.odt:
1. The table. Each condition, and the score out of 9.
2. The worst failure, explained. Using Lessons 4.2–4.5: what might the model be looking at that changes under this condition? (Pixels, brightness, background, edges.)
3. The fix. What photos would you add to training to fix it? Be specific: how many, taken where, in what light.
4. The warning label. One sentence a user must read before trusting this model in Kashmir.
Your own line (2 min, alone)
Each person writes one condition the group did not test that would matter to a real user in the valley, and why. Sign with your roll number. This part is marked for you alone.
How it is marked
Each row is marked 0–3. Rows 1–5 are shared by the group; row 6 is yours alone. Marks are returned privately and never compared with anyone else's.
Total: 18. A model that breaks badly and is honestly reported scores as highly as one that survives everything.
1. It works
- 0No model
- 1A model, not tested on ordinary new photos
- 2Tested on ordinary new photos
- 3Tested, with its ordinary score recorded as the baseline
2. Five conditions
- 0Fewer than 3
- 13–4 conditions
- 25 conditions, 9 photos each
- 35 conditions, including one the group invented
3. The worst failure
- 0Missing
- 1Named
- 2Named, with a confusion matrix
- 3Matrix, and a reason using pixels, light, edges or background
4. The fix
- 0Missing
- 1"More photos"
- 2Which photos, how many
- 3Which photos, how many, and how you'd check the fix worked
5. The warning label
- 0Missing
- 1Vague
- 2Specific
- 3Specific, honest, and useful to a real user
6. The untested condition
Just you- 0Missing
- 1A condition, no reason
- 2A real condition and why it matters
- 3A condition from real life in the valley that others missed