LLM cost is driven by tokens — the input you send plus the output you get back — priced per million tokens. You estimate spend as roughly *requests × tokens per request × price*. At scale, the bill is dominated by a handful of levers: model choice, context size, caching, and routing. Getting those right often cuts costs by more than half with no drop in quality.
The good news: you rarely need to optimize until you're actually at volume — but you should measure from day one so you know where the money goes.
How token pricing works
- A token is roughly ¾ of a word; "hello world" is about 2 tokens.
- You pay for input tokens (your prompt plus any context you attach) and output tokens (the model's response).
- Providers price per million tokens, and output is usually more expensive than input.
- Long context inflates cost fast — big system prompts and lots of retrieved RAG chunks are billed as input on every call.
Estimating your monthly cost
Before optimizing, get a rough number. The formula is simple:
// Prices are per 1M tokens — plug in your provider's current rates.
const inputPrice = /* $ per 1M input tokens */;
const outputPrice = /* $ per 1M output tokens */;
const requestsPerMonth = 100_000;
const avgInputTokens = 1_200; // prompt + retrieved context
const avgOutputTokens = 400;
const monthlyCost =
(requestsPerMonth * avgInputTokens / 1_000_000) * inputPrice +
(requestsPerMonth * avgOutputTokens / 1_000_000) * outputPrice;The levers that actually cut cost
- Right-size the model: route easy requests to a cheaper, faster model and reserve the top-tier model for hard ones — see choosing between LLM providers.
- Cache repeated work: cache answers to common questions, and use provider prompt caching for the static part of your prompt.
- Trim the context: retrieve fewer, tighter chunks and keep system prompts lean — every token of context is billed on every call.
- Cap output: set a sensible max on output tokens so responses don't run long unnecessarily.
- Batch and pre-compute where latency allows, instead of calling the model live for everything.
Prompt caching is the underrated win
Many providers cache a repeated prompt prefix (your system prompt, instructions, few-shot examples) at a steep discount on subsequent calls. Structure your prompt so the large static part comes first and the small dynamic part comes last, and input costs can drop dramatically at scale.
Don't optimize prematurely
The most expensive mistake is optimizing before you measure. Log tokens per request and cost per feature first, find the one or two lines that dominate the bill, and fix those. A dashboard of token spend by feature is worth more than a dozen micro-optimizations. This is exactly the observability we build into production AI features.
How we keep AI features affordable
We measure token spend per feature, route each request to the cheapest model that clears the quality bar, cache aggressively, and keep context tight — the same approach behind our AI chatbot builds. Book a free discovery call to size and control your AI costs.