Computer Science and Digital Literacy · 35 minutes
Suggested grades 9–12 · Unit 3: Digital evidence and privacy decisions · Lesson 1 of 2
Evaluate a Classifier’s Accuracy Claim
Read a fictional technical report and distinguish a narrow test result from a universal guarantee.
Mr. S Teaches · Original starter lesson · HS-CS-05 · Grade range is planning guidance.Watch, notice, discuss
Video choices for this lesson
Preview a video, then choose what fits your learners. “Show to class” opens a larger view; use the player’s play and full-screen controls when you’re ready.
Teacher preview needed. Check the language, level and length before showing. YouTube may show ads or related videos.
Machine Learning & Artificial Intelligence: Crash Course Computer Science #34
By CrashCourse
Introduce classifiers and predictions, then evaluate the supplied test-method evidence.
Open on YouTube: Machine Learning & Artificial Intelligence: Crash Course Computer Science #34 (opens a new tab) ↗Big Data Problems: Crash Course Statistics #39
By CrashCourse
Discuss bias and transparency as context for revising an overstated accuracy claim; use the lesson for numerical evaluation.
Open on YouTube: Big Data Problems: Crash Course Statistics #39 (opens a new tab) ↗What test evidence would you need before trusting the accuracy claim?
An external player loads only after you choose Preview or Show to class. Playback connects to YouTube. Videos belong to their creators; availability and embedding can change.
Learning objective
Assess claim support and write a justified revision using test-method details.
Before you begin
- Percentages
- Claims and evidence
Gather your materials
- Paper and pencil
- The fictional report,method card and audit note supplied here
Language to know
- classifier
- A procedure assigning items to categories.
- test set
- Examples used to evaluate a system’s outputs.
- generalization
- Applying a result beyond the examples or conditions tested.
Understand the concept
Fictional sorting-tool report:“Our system sorts every scanned label correctly.” Method card:100 clear,typed labels were selected by the developer;99 were classified correctly. No handwriting,damaged labels or unfamiliar layouts were tested. Audit note: the single error involved an unusual abbreviation. The 99% result describes this supplied set,not every future label. Selection,coverage and failure cases matter as well as the percentage.
Teach it step by step
- Separate the universal marketing claim from the recorded 99-of 100 observation.
- Identify who selected the examples and which types were absent.
- Explain why the abbreviation error challenges an absolute guarantee even before considering broader inputs.
- Write a revised technical conclusion citing specific report details and propose a more informative test.
Make the thinking visible
Worked example or model response
“The tool classified 99 of 100 developer-selected clear,typed labels correctly. This supports performance on that test set,but the abbreviation error disproves perfect classification there,and untested handwriting or damage prevents a broader guarantee.”
Guided practice
Assess a second fictional claim:“It works equally well on every layout,”supported only by 20 tests using one layout. Name the gap.
Independent practice
1. Write a three-sentence evaluation of the first report with one strength,two limitations and a follow-up test. 2. Explain whether testing 1,000 more examples of the same easy type alone fixes the coverage problem.
Quick assessment
Why should failure cases be reported rather than removed from an accuracy calculation?
Answer key & assessment criteria
Answers and teaching guidance
- Guided: The evidence covers only one layout,so equal performance across layouts is unsupported.
- Independent criteria: Accurate 99/100 statement,recognition of selection and coverage limitations,and a test including predefined varied label conditions. More easy examples improve precision for that type,not coverage of absent types.
- Quick check: Removing genuine errors inflates the estimate and hides the conditions under which the system fails.
Look for: Require the learner to explain the interpretation and check, not only state a result. Assess the written trace or evidence record; do not request real passwords, accounts or personal data.
Make it work for your learner
More support
Use a claim/evidence/method/limitation organizer and underline absolute words.
Extension challenge
Draft a testing plan separating development examples from independently chosen evaluation examples.
Verified references · Selected contributions
Standards connections
Subject-content alignment not yet verified
Supporting literacy connections
These records apply to the specified reading, writing or communication task. They do not establish subject-content alignment.
Supporting Common Core connection
RST.9-10.8
Common Core · Grades 9–10
Evaluate how reasoning and evidence support a technical performance recommendation.
Read the official standard ↗Supporting grades 9–10 technical-literacy practice in the stated task, not a verified computer-science content alignment or full text-complexity assessment.
Verified 2026-11-04. Selected contribution, not full attainment.