How we built an AI tutor that cites its sources

A study tool has an unusual failure mode. If a chatbot invents a restaurant recommendation you find out at dinner. If a study tool invents a definition, you memorise it, repeat it in an exam, and never find out at all.

So the first thing we decided about Socrate was not which model to use. It was that no answer ships without a pointer to the page it came from.

Retrieve, then answer

When you upload a PDF we split it into passages that respect the document's own structure — a section heading starts a new passage, a table stays whole — rather than cutting every N characters. Each passage keeps its page number and heading path.

A question is answered in two stages. We first find the passages most likely to contain the answer, then hand only those to the model along with an instruction to answer from them and nothing else. If the passages don't contain an answer, the correct output is to say so.

Making 'I don't know' cheap

Models are trained to be helpful, which under pressure means guessing. Most of our prompt work went into making abstention a normal, low-cost outcome instead of a failure — and into checking, after generation, that every claim in the answer is actually supported by a retrieved passage. Claims that aren't get dropped before you see them.

An answer you can check in ten seconds is worth more than a better answer you have to trust.

Citations as an interface, not a footnote

Every sentence in an answer links to the passage behind it, and clicking it scrolls the PDF to that spot with the passage highlighted. This does two things: it makes verification fast enough that people actually do it, and it turns the citation into a way to read further rather than a disclaimer nobody clicks.

It also keeps us honest. When the retrieval layer is weak, bad citations are obvious to every user immediately — which is a much better bug report than a plausible-sounding paragraph.