LLM Inference Playgrounds

Retrieval decides what goes into the prompt. Inference is what happens next: a model on a GPU reads the prompt and writes the answer, one token at a time. These seven playgrounds follow one question from start to finish: what limits a GPU, and what does each technique buy you?

Start here: New to this? Read the two-minute overview first: it shows one request passing through a GPU and where each idea fits. How LLM inference works

Step 1Prefill vs DecodeWhy the first token and the next hundred are limited by different things.Main control: Prompt length and batch sizeOpen →Step 2KV Cache CalculatorWhy long context and big batches fill GPU memory long before the weights do.Main control: Context length and batch sizeOpen →Step 3Quantization VisualizerWhat INT8 and INT4 do to weights, and why block-wise scaling exists.Main control: Bits and group sizeOpen →Step 4Batching SimulatorStatic batching, continuous batching and paged KV memory, running side by side on the same requests.Main control: Arrival rate and KV memory budgetOpen →Step 5Prefix Caching SimulatorHow much of a prompt a cache can reuse, and why routing by prefix matters.Main control: Shared prompt, chunk repeat and routingOpen →Step 6 · optionalSpeculative Decoding SimulatorWhen a small draft model plus one checking pass beats plain decoding.Main control: Draft length k and acceptance rateOpen →Step 7LLM Inference Cost CalculatorWhen running your own GPU beats paying an API per token, and why utilization decides it.Main control: Utilization and monthly tokensOpen →

The pages are numbered in reading order, and each one ends by handing you to the next, carrying your numbers across where they apply. Each page is a small calculator or simulator with a "Show me" walkthrough. Nothing is installed and nothing is uploaded. Settings are kept in the address bar so you can share them, and every page exports its numbers as CSV or JSON.

Where RAG comes in: retrieved chunks make every prompt longer. On the pages where the prompt length matters, tick "Add retrieved chunks" to see what top-k and chunk size do to memory, batching and cost.

Guides

How LLM inference works · What is the KV cache? · What is quantization? · Continuous batching, explained · Speculative decoding, explained · Self-hosting an LLM or using an API: what it costs · Prefill and decode, explained · Prefix caching, explained