Problem
A wrong answer should tell a teacher why, not just who
Most quizzes tell a teacher which students got a question wrong. They rarely say why. Research on diagnostic multiple choice (Sadler, 1998; Briggs et al., 2006) shows that each wrong option can be written to catch one specific misconception. Writing those options by hand is slow, which is exactly where AI drafting could help, if teachers can check and correct what it writes.
CHECKPOINT had two users from day one: teachers, who need a quiz they trust in minutes, and the research team, who need evidence from real classrooms, not demos.
Decisions
Four changes, each traced to what we heard
- Teachers reported shallow coverageTeachers who picked three misconceptions got three questions that each tested only one. I rebuilt generation around answer-level diagnosis: every wrong option is tagged with the misconception it reveals. The model may suggest more, shown with a violet “AI-suggested” chip that teachers can change.
- An advisor asked for a calmer modeAdded Simple view: large text and three big buttons, for teachers who want to launch a quiz and nothing else.
- New teachers couldn’t picture the payoffTeachers without a class had no way to see a report. A practice run with six simulated students produces a real report and writes nothing.
- The research team wanted to test themesBuilt a balanced K-12 sample of matched question pairs, with and without an engagement theme, for an expert review survey.
Instruments
Research built into the product, not bolted on
- An event ledgerAn append-only record of every question shown, answer saved, and revision, with time on question, exit reason, and both client and server time. AI generations are logged by hash, not by content.
- Pre/post studies that state their limitsTeachers can run a pre/post study with parallel forms. The tool computes gains and effect sizes, checks statistical power before the study starts, and says on screen what the design cannot prove.
- A model comparison consoleResearchers compare up to four model or prompt variants on real curriculum cases, side by side, with output validity rate, p50/p90 latency, and cost.
- Privacy by defaultStudents join with a code and a nickname, no account. Analytics stay off until consent and retention policies are in place.
Impact
Where it stands
CHECKPOINT won a Catalyst Prize in the Learning Engineering Tools Competition and is live at checkpoint.viablelearning.org, with exports to PDF, Excel, and Canvas. It is still in pilot, so I am not claiming adoption numbers yet. The instruments above are what will produce them.
I built this as the only engineer, and direct teacher fieldwork was led by a teammate. The research plumbing is ahead of the evidence. The next step is the pilot data itself, starting with two questions: how often do teachers keep or override AI-suggested misconceptions, and do generated wrong answers actually attract the students they predict?