LLM Inference Cost Calculator
Compare the cost of running a model on your own rented GPU with paying an API per token, and see how much utilization decides it. All figures are estimates. Nothing is uploaded.
- 1Pick a GPU and enter its tokens per second
- 2Set the utilization or the monthly tokens
- 3Read the break-even point against an API
Self-hosting
Throughput helper: estimate the tokens per second
This is a ceiling set by memory speed, not a measurement. Real servers are slower. The Batching Simulator shows how requests are packed together.
How much it is used
API to compare with
Assumptions
A month is counted as 730 hours. The GPU is rented all month and is never switched off.
Estimate only. List prices and a throughput you entered; your real bill will differ.
Monthly cost against monthly tokens
Cost per 1M tokens at each utilization
| Utilization | Self-hosted | API | Cheaper | Self-hosted ÷ API |
|---|
How each figure is worked out
What it is for
- Work out what a self-hosted model costs per million tokens at a given utilization.
- Find the monthly volume at which renting a GPU becomes cheaper than an API.
- See why an idle GPU makes self-hosting expensive, and what batching does to the price.
- See what longer RAG prompts do to the comparison.
How it works
An API charges for each token. A GPU charges for each hour, whether it is working or waiting. That one difference decides most self-hosting questions.
The self-hosted cost per million tokens is the hourly GPU price divided by the tokens actually produced in that hour: tokens per second × 3,600 × utilization. Utilization is the share of the time the GPU is busy. At 100% the GPU is as cheap per token as it can be. At 10% every token costs ten times as much.
The API side is the output price, plus the input price multiplied by the number of prompt tokens sent for every answer token. With RAG the prompt is several times longer than the answer, so the input price matters more than it first appears.
The break-even point is where the monthly GPU rent equals the API bill for the same number of tokens. Below that volume the API is cheaper. Above it self-hosting is cheaper, until the GPU is full and a second one is needed, which is the step in the chart.
This is a planning estimate. It leaves out the cost of the people who run the servers, storage and network charges, the time a GPU spends starting up, reserved-instance and spot discounts, and API discounts such as prompt caching and batch processing. Self-hosted throughput is counted in generated tokens; reading the prompt is assumed to be included in it.
Questions
Is self-hosting an LLM cheaper than an API?
Only when the GPU is kept busy. A rented GPU costs the same every hour, so its cost per token depends on how many tokens it produces. Below the break-even utilization shown on this page, the API is cheaper.
What is utilization?
The share of the time the GPU is producing tokens at its full rate. A service used mostly in office hours in one time zone is often below 30% across a whole month.
Where do I get the tokens per second?
Measure it if you can: it depends on the model, the GPU, the inference server and above all the batch size. The throughput helper gives an upper limit worked out from the GPU memory bandwidth, and the Batching Simulator shows why serving many requests at once raises it.
Why does a longer prompt make the API dearer but not the GPU?
An API bills every prompt token on every request. On your own GPU a longer prompt takes more time and memory, which lowers the tokens per second, but there is no separate charge. Enter a lower throughput to allow for it.
What does the throughput helper calculate?
To produce one token, the GPU has to read all the model weights and the cache for that conversation from its memory. Dividing the memory bandwidth by the bytes read gives the most tokens a second one request can get. A batch reads the weights once for all its requests, so the total rises with batch size.
Where do the prices come from?
GPU prices are one provider's public on-demand list prices on the date shown under Assumptions, and API prices are the same table the Token Counter uses. Both change often. Choose Custom to enter your own.
Can I share my settings?
Yes. The settings are kept in the page address, so Copy link gives a link that opens the page exactly as you left it. CSV and JSON download the inputs and the results.
Is anything uploaded?
No. The page is a calculator that runs in your browser tab. The numbers you enter stay on your device.