Knowledge gaps and the teach loop: why an AI agent's wrong or missing answers are a list, not a failure, how the list is produced, and the ten-minute weekly routine that turns it into a better agent
New owners expect an AI agent to know everything on day one and are disappointed when it says "I'm not sure" on day two. The "not sure" is the feature. Each one is a logged question with a missing answer, and the business that spends ten minutes a week answering them has an agent that knows more every Monday than it did the last.
Quick answer
A knowledge gap is a customer question the agent could not answer from your knowledge base, so it said it was not sure and offered a person instead of guessing. The agent records each gap. The teach loop is the weekly routine: open the gap list, write the answer for each recurring question as a line in the knowledge base, retest, and watch the list shrink. Most businesses see gaps fall to a trickle within four to six weeks. The list is also market research, because it is exactly what customers wanted to know and could not find. ReplyKit shows the gap list per agent in the dashboard.
What a gap is, and is not
- A gap: "Do you do gel removal without a new set?" when your price list only lists gel sets. The agent says it is not sure and offers a person. Logged.
- Not a gap, a wrong answer: the agent quotes last year's price because both prices are in the knowledge base. Not logged as a gap, because the agent thought it knew. Found by reading conversations, and fixed by removing the old line; see why the agent answered wrongly.
- Not a gap, a hand-over: a complaint. Routed to a person by rule, not for lack of knowledge.
- The agent is instructed to prefer "not sure" over guessing, which is why gaps exist and hallucinations are rare; the mechanism is in why AI agents hallucinate.
The weekly loop
- Open the gap list. Group near-duplicates ("gel removal", "take off gel", "remove old gel").
- For each recurring question, write the answer as one to three plain lines in the knowledge base. If the answer is "no, we don't", write that too; a clear no is an answer.
- For questions you do not want the agent to answer (custom quotes), add a hand-over rule instead.
- Retest the question in the test chat.
- Ten minutes. Done. The format rules are in what to put in the knowledge base.
How long it takes
| Week | Typical gap volume | What is happening |
|---|---|---|
| 1 | High; many recurring | The knowledge base reflects what you thought customers ask, not what they ask |
| 2 to 3 | Falling fast | The top twenty real questions are now answered |
| 4 to 6 | A trickle | Long-tail questions; most are one-offs |
| Ongoing | Spikes when something changes | A new product, a price change, a holiday |
A spike after a change is normal and is the signal to update the knowledge; the first month is described in the first 30 days with an AI agent.
The gap list as market research
The list is a record of what customers wanted and could not find, uncontaminated by what you assumed. Three people a week asking about a service you do not offer is a product decision. Repeated questions about parking mean your website lacks a line. Questions about a competitor's feature tell you what they are comparing. Review it in the monthly audit for that purpose, separately from the weekly fix.
Frequently asked questions
What is a knowledge gap?
A customer question the agent could not answer from your knowledge base, so it said it was not sure and offered a person. Each is logged.
Is a gap the same as a wrong answer?
No. A gap is a known unknown; a wrong answer comes from wrong or conflicting knowledge and is found by reading conversations.
How often should I review gaps?
Weekly, for ten minutes. Answer the recurring ones, add hand-over rules for the ones you do not want answered.
How long until gaps are rare?
Four to six weeks for most businesses, with spikes whenever something changes.
What else is the gap list good for?
Market research: it is what customers wanted to know and could not find.
Smarter every Monday
The gap list in your dashboard, the answers in your words, the agent retested in seconds. 7-day free trial, no card.
Sources and further reading
- Harvard Business Review, The Short Life of Online Sales Leads (Oldroyd, McElheran, Elkington, 2011) · firms that responded within an hour were about seven times more likely to qualify the lead
- WhatsApp Business Messaging Policy · opt-in, opt-out and prohibited-use rules every business must follow
ReplyKit is an independent product and is not affiliated with or endorsed by Meta or WhatsApp. This article is general guidance for running a small business, not legal, financial or regulatory advice.