AI features are cheap to prototype and surprisingly expensive to run. The bill grows with every user, every long prompt and every retry. Most of it is avoidable without making the product worse.
1. Measure cost per feature
You can't cut what you can't see. Log the model, input tokens and output tokens for every call, tagged with the feature that made it. One feature is usually responsible for most of the spend.
2. Use the smallest model that passes
Classification, extraction and routing rarely need the biggest model. Build a small test set of real inputs with known good answers, then try cheaper models against it. Keep the large model for the steps that genuinely need it.
3. Cache what repeats
Most providers offer prompt caching: a long, unchanging prefix such as instructions or a reference document is billed at a fraction of the price on repeat calls. Put stable content first and variable content last to benefit. For identical questions, cache the whole answer in your own database.
4. Batch what isn't urgent
Overnight reports, bulk document processing and back-fills don't need instant answers. Batch APIs from the major providers typically cost around half the price of real-time calls.
5. Send less context
Dumping whole documents or full chat histories into every prompt is the most common source of waste. Retrieve only the relevant passages, summarize old conversation turns and cap output length.
6. Stop runaway loops
Agents and retries can call a model dozens of times for one request. Set limits on steps, retries and spend per user, and alert when a single request crosses a threshold.
Deeraf builds and audits AI features with costs visible from day one. A Tech Check includes AI and cloud spend, with the savings priced out.