AI study tools for medical students: the strict-scope problem
The default LLM behavior is the problem
If you paste a transcript into ChatGPT and ask for a study guide, it will write a beautiful one. It will also include teaching points, mnemonics, and "high-yield" facts the lecturer never mentioned. The model is doing what it was trained to do: produce a complete, plausible-sounding artifact. Completeness is rewarded; fidelity to the input is not.
For board prep, this is dangerous. Every drilled card that doesn't reflect what your professor emphasized is a wasted rep, and worse, it false-confirms a gap you don't actually have.
Strict-scope as a constraint, not a vibe
The fix is to scope-lock the prompt. Not with a polite "please don't make things up" (that doesn't work), but with hard rules:
- The transcript is the source of truth. Every fact in the output must derive from the transcript. If a fact isn't there, omit it instead of filling.
- Reference real artifacts only. Screenshots are referenced by ID from a candidate list; the model can't invent IDs. AnKing tags are vector-matched to a live database; the model can't invent tags.
- Under-coverage is a bug; over-coverage is a worse bug. A 90-minute lecture warrants a long guide, a 20-minute lecture a shorter one. Padding to hit a target length dilutes what the lecturer actually taught.
What this looks like in practice
SlydeMe's study guide is shorter than what generic AI tools produce. The Anki gap-fill deck is smaller. The AnKing tag list is more precise. That's because we cut every line that the lecturer didn't actually deliver. The model wants to add Quincke pulse to your aortic regurgitation guide because it's "high-yield." If your lecturer didn't say it, we cut it.
Less impressive on a marketing slide. More useful on exam day.