The method
Answer only from your own documents, and show the source
Tell the assistant to answer only from the documents you give it, and to show which document each answer came from. A reader can then check the source in seconds. An answer with no source is a warning sign.
How to check it: Pick five answers at random and open the source each one cites. Does the source really say that?
Let it say "I don't know"
An assistant that must always answer will sometimes invent. Give it a clear way out: if the documents do not contain the answer, it says so and points the person to a human.
How to check it: Ask it five questions your documents do not cover. It should decline all five.
Don't let the AI decide what is right or wrong when software can check
Where there is a known correct answer, such as a date, a figure or a name, use simple software to compare the reply with it. Software does not get tired or generous. Keep AI judges for the things software cannot check, and do not trust them alone.
How to check it: List the questions that have one known answer. Write the answers down and compare replies to them automatically.
Test on a fixed set of real messages, typed the way people really type
Real users write in half sentences, with typos, with no punctuation, and with the wrong word for things. Collect real messages and keep them as a fixed test set. Run the same set every time you change anything, so you can see what got better and what got worse.
How to check it: Start with 100 real questions from your team or customers. Do not tidy them up.
A person reads every reply before each release
Software catches what it was told to look for. A person catches the rest: wrong tone, half-right answers, answers that sound sure and are not. Before each release, someone reads every reply on the test set.
How to check it: Give the reader a simple sheet: right, wrong, or "I don't know", plus a note.
Measure both mistakes: made-up answers and wrong "I don't know" replies
Counting only made-up answers rewards an assistant that refuses everything. So count two things: replies that invent an answer, and replies that say "I don't know" when the documents did have the answer. Push down one and the other tends to go up, so you need both numbers.
How to check it: Report both counts on every test run, side by side.
What this looked like in practice
We used this method on an AI tutor for an education company in a regulated industry. Across 150 test messages, each read by a person, we saw no made-up answers. Wrong "I don't know" replies, where the assistant refused a question it could have answered, went from 6 to 0 on the 35 test messages about terms in the material. And the AI fees were $2.52 for 684 AI answers, about $0.004 each.
That is one assistant on one set of documents, not a guarantee for yours. It is why we test every assistant on the customer's own documents before anyone relies on it. The full write-up is in the AI tutor case study.
The short version to follow
- Restrict the assistant to your documents and show the source.
- Allow "I don't know" and send those people to a human.
- Check by software wherever a known answer exists.
- Test on a fixed set of real messages.
- Read every reply on the set before each release.
- Count made-up answers and wrong "I don't know" replies.
Next step
Want to see how an assistant behaves on your documents? Hundred Questions is a one-week test: we build it, your team sends 100 real questions, a person checks every answer.