Debug RAG: why did it give a wrong answer?

Bring your document, the question, the wrong answer you got and the right one. See where each answer sits in the document, what four different settings retrieve, and which step most likely went wrong. Nothing is uploaded.

Your document

The question and the answer you got

Your settings

Set these to match your system, so the first column behaves like it. The other three columns change one thing each.

Optional. With a model connected, each of the four settings also writes an answer from the chunks it retrieved, so you can see which settings give the right answer. Requests go straight from your browser to this address.

Examples

What it is for

How it works

A wrong answer from a RAG system has one of a few causes. The answer may not be in the document at all, or its text was lost when the file was converted. The chunking may cut it in two. Search may not retrieve it, because the question uses other words or another passage scores higher. Or it was retrieved and the model still answered from something else.

This page works through those in order. It finds the passage in your document that holds the correct answer, and the passage your wrong answer most resembles: very often the model answered faithfully from the wrong chunk. Then it retrieves with your settings and with three others that each change one thing a lot (small chunks, whole sections, and hybrid search or more chunks), and shows where each passage landed.

It imitates your system with its own chunking and search, so it points to the likely cause, not a certainty. When your settings here match your system, the first column is close to what your system did. Everything runs in your browser.

Questions

Why do I need the correct answer?

To find the passage that should have been retrieved. Without it the page can still show where your wrong answer seems to come from, but it cannot say whether the right passage was missed.

Does it run my RAG system?

No. It runs its own chunking and search on your document with settings you choose. Set chunk size, overlap and top-k to match your system so the first column behaves like it. To score what your own system actually retrieved, use Evaluate your RAG output.

What does "the passage your answer seems to come from" mean?

The sentence or two in your document that share the most words with the wrong answer. If it is a different passage from the right one, your system most likely retrieved the wrong chunk and the model answered from it.

Do I need an AI model?

No. The diagnosis needs no model. Bringing your own model (BYOM), such as Ollama, Gemini or Claude, adds the answer each of the four settings would give: four calls per question.

Is my text uploaded?

No. Everything runs in your browser tab. Your documents and questions never leave your device.