JPMorgan Chase researchers have tested a way to find evidence that a retrieval-augmented AI system leaves out of its answers. Their method, called Re:CAP, checks whether the search stage missed relevant material already in its document collection. The team described it in a paper first posted September 21.
Retrieval-augmented generation, or RAG, lets an AI assistant search a document collection before answering a question. An answer can faithfully cite the documents it found while still missing a document that would change the answer. Re:CAP starts with that first answer and its retrieved documents. It lists the topics they cover, generates follow-up search questions about possible gaps, and uses an AI judge to keep newly found documents that add relevant information. It is an audit layered on the existing search system, rather than a replacement for it.
On MuSiQue, a public test of questions requiring multiple pieces of evidence, the audit recovered 90.1% of labeled supporting documents while examining an average of 238 distinct retrieved documents per question. A flat hybrid search examined 500 documents and reached 77.2% recall. The 12.9 percentage-point difference measures found evidence under those test conditions, not the quality of final answers. In two proprietary production systems, the researchers also ran the audit on 200 sampled questions each, but those collections lack the labels needed to calculate the same recall comparison.
More searching did not always improve the headline metric. On TREC-COVID, a dataset with many relevant documents per question, Re:CAP's recall was 22.1%, below a flat BM25 search's 24.1%. Yet 21.2% of the relevant documents Re:CAP did find were absent even from the combined top 500 results of three search methods. Across TREC-COVID and MuSiQue, human reviewers judged 97 of 123 newly surfaced documents to contain information missing from the original answer.
The method is too costly to run on every user question: the paper reports 250 to 500 AI model calls per audited question. Its proposed use is a periodic sample audit, especially before a search index or ranking change. It can expose missing material in an existing collection, but cannot find evidence the collection itself never contained. The experiments were in English, and the paper does not establish that the same measured recall gains will transfer to other production systems.
Sources
- Joshi et al., “Re:CAP – Auditing Retrieval Coverage in Production RAG Pipelines,” arXiv, version 2 (September 22, 2026)
- Full paper, methods, results and limitations
- Lead image: generic RAG workflow diagram by Numiri, Wikimedia Commons, licensed under CC BY 4.0. It illustrates a standard workflow, not Re:CAP's audit.
