Automate drafts, not ownership

Problem

Students spend their energy making study materials, then run out before studying

Learners know retrieval practice works. The problem is the setup: turning messy lecture notes, PDFs, and links into usable flashcards takes so long that review never happens. One medical student in our study spent three of his four exam-prep months building flashcards and never finished reviewing them.

Generative AI can draft those cards in seconds. The open question for the product was not can we automate it, but which parts should stay with the learner, because checking, editing, and choosing what to study are themselves part of learning. (The paper calls the prototype SmartFlash.)

Approach

A working prototype, studied four ways

We studied a working prototype. Learners bring their own material, the AI drafts flashcards from it, and a study path suggests what to review next. Then four steps, each chosen to catch what the previous one would miss:

• LLM-seeded hypothesesAsked an LLM to predict student needs and breakdowns. Fast and cheap, and it gave us a baseline to test AI-generated assumptions against real people.
• Cognitive walkthroughsTwo researchers stepped through the prototype to find friction, missing feedback, and places where automation might overreach, before involving participants.
• Think-aloud + interviewsSix students (ages 22–31, five disciplines) used the prototype with their own course materials while thinking aloud. Real materials surface breakdowns that demo content hides.
• Reflexive thematic analysisHuman-led coding (Braun & Clarke) connected moment-to-moment behavior to cognitive, metacognitive, agentive, and affective themes.

Findings

From what people said to what the product does

Pick a finding to follow it from a participant’s words to the design decision it produced.

LLM vs. people

Can an LLM replace early user research? It found the what, and missed the why

We compared the LLM’s predicted needs with what participants actually did and said. The model correctly named the functional problems: preparation takes too long, and it is hard to prioritize what to study.

It consistently missed the affective and agentive ones: the anxiety underneath the overwhelm, the need to edit and own AI output, and the way the same leaderboard motivated some students and shut others down. LLM-generated hypotheses are a useful starting point. Lived experience still needs human-centered research.

Impact

Four design principles for AI study tools

• Automate drafts, not ownershipAI removes the blank-page work; learners stay responsible for checking and shaping every card.
• Make uncertainty visibleRecommendations say what they are based on and how to override them.
• Support multiple study moodsPractice, paths, and decks are adjustable, so the product serves both high-energy review and low-effort maintenance.
• Keep the learner in the loopThe question is not how much can be automated, but which parts of self-regulation should stay visible and practiced.