Guides
Short, plain-language guides to the ideas behind retrieval-augmented generation, and to how a model is run once the prompt is built: memory, quantization, batching, speed and cost. Each one links to a tool on this site where you can see the idea working.
New to RAG? Follow the learning path: eight short steps through these guides and the tools, in reading order.
RAG basics
- What is RAG?
What retrieval-augmented generation is, how it works step by step, and when it is the right choice. - What are embeddings?
How text becomes a list of numbers, and how two lists are compared to find similar chunks. - What are tokens, and what do they cost?
What a token is, how to estimate tokens from words, and how to work out the cost of a prompt. - How to choose a chunk size for RAG
How chunk size, overlap and splitting method change what a RAG system finds, with starting points to try.
Better retrieval
- Hybrid search: words plus meaning
Why keyword search and meaning search fail in opposite ways, and how combining them finds more. - Re-ranking: a second, more careful look
Why search results are scored a second time, and what a re-ranker changes. - How to test RAG retrieval
Build a small evaluation set, measure hit rate and MRR, and compare settings fairly, with a worked example. - RAG auto-tuner: find the best settings for your documents
Tries chunking, search method, top-k and more on your questions and recommends a setup, with the reasons. - Seven common RAG mistakes
The most frequent reasons RAG gives poor answers, each with a practical fix. - What is GraphRAG?
How a knowledge graph helps answer questions that span several chunks, with a worked example and a playground to try it.
How models run
- How LLM inference works
One request through a GPU in one picture, and where KV cache, quantization, batching, speculative decoding and cost each fit. - Prefill and decode, explained
Why the first token and the hundred after it are limited by different things. - What is the KV cache?
What a model stores while it writes, how to work out its size, and why long context and big batches fill a GPU. - What is quantization?
How a model is stored in 8 or 4 bits, where the error comes from, and how to choose a GGUF quantization. - Continuous batching, explained
How one GPU serves many requests at once, and why refilling the batch every step makes such a difference. - Prefix caching, explained
What a server can reuse between requests, and what that means for a RAG prompt. - Speculative decoding, explained
How a small draft model lets a large one write several tokens per pass, and when it does not help. - Self-hosting an LLM or using an API: what it costs
How to compare a rented GPU with an API price, and why utilization decides the answer.
Practice
- RAG interview preparation: questions you can try
Eleven common AI engineer interview questions on retrieval, evaluation and serving, each with a short answer and a tool to try it.