Check RAG citations and fact-check AI answers

Retrieval-augmented answers look trustworthy because they carry citations. Agentlab checks whether the citations actually say what the answer claims. Open the RAG assistant: grounding and cost template.

1. RAG answer with citations

Select Leave policy answer. The check needs two inputs:

• Generated answer with citation markers — markers such as [1], [doc2] or [leave-1];
• Retrieved chunks — JSON like [{"id":"leave-1","text":"…"}], or one chunk per block starting with [id].

The RAG Citation Debugger compares every sentence with the chunks it cites. The result: 3 of 5 claim sentences are supported by their citations (60 %). The table Sentences shows a verdict per sentence:

• Number mismatch — sentence 2 says up to 10 unused days while chunk leave-2 says 5;
• Uncited claim — Unused days expire at the end of March has no marker (the fact is in a chunk, but the reader cannot check it);
• Wrong citation — the sentence cites a chunk that does not support it; the column Better chunk suggests one when another chunk fits.

The table Chunks shows which retrieved chunks were never cited: a hint that retrieval returns more than the model uses.

Support threshold controls how many of a sentence's content words must appear in the cited chunk (default 0.5). Raise it for stricter checks.

2. Answer fact check

Select Revenue statement in the newsletter. The Answer fact check splits an answer into claims and searches your Source texts (separate several sources with a line that contains only ---) or up to five Source URLs.

The claim revenue of 12 billion euros in 2025 is flagged as Number contradicts the source: the press release says 9.4 billion. The gate Numbers that contradict the sources ≤ 0 fails.

3. Gates for grounding

Two gates fit most RAG products:

• RAG claims supported by their citations ≥ 80 %
• Answer claims supported by the sources ≥ 80 %

Collect several answers as separate checks (Ctrl+D duplicates the selected check); the worst one decides the gate.

4. Limits

Both tools work with word overlap and number matching, not with a language model. They catch wrong numbers, misattributed citations and missing support reliably, but they cannot judge paraphrases that use completely different words, and a supported sentence can still be misleading. Use the findings to decide where a human should read closely.