Keep OpenAI API Costs Under Control in Your App

This post is for people who have already shipped an AI feature and want to keep the bill predictable as usage grows. I assume your OpenAI calls run on a server (a Supabase edge function or similar), as they should. If your key is in front-end code, fix that first, because no cost control matters while anyone can copy your key.

The approach I teach is simple: measure first, then cut, then cap. Guessing at costs leads to either panic or a surprise invoice.

1. Measure cost per action

Every Responses API result includes a usage object with input_tokens and output_tokens, plus details like how many input tokens came from the cache. Log those for every call, along with the user, the feature and the model. Here is a Supabase table for it:

create table ai_usage (
  id bigint generated always as identity primary key,
  user_id uuid not null references auth.users(id) on delete cascade,
  feature text not null,
  model text not null,
  input_tokens int not null,
  cached_tokens int not null default 0,
  output_tokens int not null,
  created_at timestamptz not null default now()
);

alter table ai_usage enable row level security;

create policy "Users can read their own usage"
  on ai_usage for select
  using (auth.uid() = user_id);

And in your edge function, after a successful call:

await admin.from("ai_usage").insert({
  user_id: user.id,
  feature: "summarize-note",
  model: MODEL,
  input_tokens: data.usage?.input_tokens ?? 0,
  cached_tokens: data.usage?.input_tokens_details?.cached_tokens ?? 0,
  output_tokens: data.usage?.output_tokens ?? 0,
});

Here admin is a Supabase client created with your secret (service role) key inside the function, so users cannot write fake usage rows themselves. Never use that key in the browser.

After a week, multiply the average tokens per call by the prices on OpenAI's pricing page. Now you know what one click costs, what your heaviest users cost, and whether your pricing covers it. That one number drives every decision below.

2. Use the smallest model that does the job well

This is usually the biggest saving. OpenAI's current lineup runs from a flagship model for hard reasoning and coding down to a small, efficient tier built for focused, high-volume tasks. As I write this, the per-token price gap between the top and bottom of the lineup is roughly a hundred times. Check the models page for current names and positioning, since they change several times a year.

Summaries, classification, tagging, extraction and short drafts rarely need the flagship. How I pick:

Collect 20 to 30 real inputs from your app (with personal details removed).
Run them through a capable model and save the outputs as your quality bar.
Run the same inputs through the next smaller model, then the next.
Stop at the smallest one whose outputs you would still be happy to show a paying user.

OpenAI's own production advice is the same: validate on a more capable model, then test whether cheaper ones are good enough. Keep the model name in one constant so switching is a one-line change.

Turn down reasoning effort

Current models accept a reasoning.effort setting. Reasoning tokens are billed as output, so a summary feature running at high effort is paying for thinking it does not need. Start at a low setting for simple features and only raise it if quality suffers.

3. Cap the output and trim the input

Set max_output_tokens on every call. It puts a ceiling on the worst case. Set it high enough that normal answers fit, or you will get cut-off responses.
Limit input length on the server. If your feature summarizes notes, nobody needs to send 50 pages. Reject or truncate long input with a clear message.
Send only what the model needs. If you include records from your database as context, send the few relevant fields, not whole rows.
Ask for short output in the instructions. "Summary of 2 to 3 sentences" costs less than an open-ended "summarize."

4. Make prompt caching work for you

OpenAI applies prompt caching automatically on supported models. When a request starts with the same text as a recent request, the repeated part is billed at a reduced cached rate. On current models, the repeated prefix needs to be at least 1,024 tokens to be cached, and newer models also bill a small extra amount when a cache entry is written. The prompt caching guide has the current rules and retention options.

What you control is the order of your prompt:

Static content first: instructions, examples, schema and tool definitions.
Changing content last: the user's text, their name, today's date.

Putting a timestamp or user name at the top of your instructions breaks caching for everything after it. Short prompts will not hit the cache at all, and that is fine. Caching matters most for features with long, fixed instructions or reference material. Check cached_tokens in your usage table to see if it is working.

5. Cache your own results

Prompt caching cuts the price of repeated input. Your own cache removes the call entirely. If the same input always produces an answer you are happy to reuse (a summary of a document that has not changed, a description for a product), store the result and return it next time.

A simple version: hash the input text, store the hash with the result in a table, and check that table before calling OpenAI. Also save results on the record itself (a summary column on the note), so users who reload the page do not pay you for the same summary twice.

6. Set limits per user

Every AI feature needs a per-user limit, even generous ones. It protects you from one heavy user, a bug that calls the API in a loop, and anyone abusing a free plan. With the usage table above, a daily cap is a few lines at the top of your edge function:

const DAILY_LIMIT = 50;

const startOfDay = new Date();
startOfDay.setUTCHours(0, 0, 0, 0);

const { count } = await admin
  .from("ai_usage")
  .select("id", { count: "exact", head: true })
  .eq("user_id", user.id)
  .gte("created_at", startOfDay.toISOString());

if ((count ?? 0) >= DAILY_LIMIT) {
  return json({ error: "You have reached today's limit. It resets at midnight UTC." }, 429);
}

Tie the limit to the user's plan if you have Stripe tiers: a small allowance on free, more on paid. Show people their remaining usage in the app. Users handle a visible limit far better than a surprise error.

Also send a hashed user ID in the safety_identifier field. If one user misuses your feature, it helps OpenAI deal with that user instead of your whole account.

7. Move background work to cheaper processing

Not every AI call needs an answer in two seconds. Nightly digests, bulk tagging, re-summarizing old records and running your quality tests can wait.

Batch API: submit many requests as a job and collect results later, at a discount compared to standard requests.
Flex processing: set service_tier to "flex" for Batch-level pricing on normal requests, in exchange for slower responses and occasional "resource unavailable" errors you need to retry. It is in beta with limited model support, so check which models qualify.

Keep user-facing buttons on standard processing. Use these for anything that runs on a schedule.

8. Put a ceiling on the whole account

In the OpenAI platform you can set spend alerts and hard spend limits for the organization and for each project. Alerts just notify you. A hard limit blocks requests once you reach it, and those requests fail with a 429 error. Enforcement is not instant, so spending can slightly pass the limit.

My setup:

A separate project for staging and for production, each with its own key and limits.
Alerts at about half and three quarters of what I expect to spend in a month.
A hard limit on staging, always.
On production, a hard limit set well above normal usage, so it only trips on a real problem. Make sure your app shows a friendly message when it does, because your AI feature will stop working until you raise the limit or the month resets.

A weekly five-minute check

Open the usage dashboard and compare this week to last week.
Query your ai_usage table for the top 10 users by tokens.
Check your cache hit rate if your prompts are long.
Ask whether any feature could drop to a smaller model.

Where to go next

The recipes at /recipes include usage tracking and limits as part of each AI feature spec, so you can add them from day one. If you want help checking your own numbers against your pricing, bring them to a cohort office hours session.