💰 Budgets Management
Budgets management for LLMs with a gateway means monitoring and managing the costs of using different LLMs through an API gateway.
Our Otoroshi LLM extension helps you optimize usage, control your budget, and improve cost efficiency across models.
Configuration
budgets {
enabled = true
enabled = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_BUDGETS_ENABLED}
embed-budgets-in-responses = false
embed-budgets-in-responses = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_BUDGETS_EMBED_BUGETS_IN_RESPONSES}
}
Once it's enabled, you can create budgets from the UI or the admin. API and enforce budget limits to any consumer
Budgets
A budget is defined between a start date and an end date. Limits are applied for a duration that is renewed between the start and end date. Limits can be expressed in USD or in tokens.

A budget can defined limits that can be global or per usage (inference, image, audio, video, embedding, moderation, decision)
Every modality feeds those counters, in tokens and in dollars: an image generation, a transcription, a voice and an extracted page are priced the way cost tracking describes, and a call is refused once the limit it falls under is reached. The only calls that move tokens without moving dollars are the ones priced in a unit the gateway cannot measure — a voice billed per second of audio.


The counters of a cycle stay available for a week after it ends, so the calls a worker of a cluster counted in its last minutes are never lost, then they are cleaned up automatically, like the counters of a deleted budget.
A budget is defined for a scope based on pretty much whatever you want (apikey, api group, user, provider, model, request, etc). You can reference a budget in entity metadata using ai_budget_ref (apikey, api group, user, provider, auth. module)

Scope options
| Scope | Description |
|---|---|
apikeys | List of API key IDs. Everything the gateway calls to answer a request counts for its caller: the answer, and with it the embeddings of a semantic cache, the model a router asks to pick a candidate, the panel and the judge of a fusion |
users | List of user emails or IDs (regexes). Calls made with an API key count for the owner of the key (its ai_studio_owner metadata) when no user is authenticated |
groups | List of API group IDs |
providers | List of LLM provider IDs. A request handed over to another provider (its fallback, the target of a load balancer, the candidate of a router) counts for the provider that received it and for the one that served it: put a budget on a load balancer to cap everything going through it, on a target to cap what that target serves. A budget with both in its scope counts the request once |
models | List of model names |
extractFromApikeyMeta | Extract budget ref from API key metadata |
extractFromApikeyGroupMeta | Extract budget ref from API key group metadata |
extractFromUserMeta | Extract budget ref from user metadata |
extractFromUserAuthModuleMeta | Extract budget ref from auth module metadata |
extractFromProviderMeta | Extract budget ref from provider metadata |
rules | JsonPath-based rules for advanced scoping |
rulesMatchMode | Rule matching mode: All (all rules must match) or Any (at least one rule must match) |
alwaysApplyRules | When true, rules are always evaluated even if explicit lists match |
partition_by | An expression of the Otoroshi expression language: the budget counts apart each value it takes, each one getting the whole limit. See budgets per consumer |
Duration units
Budget limits are applied for a renewable duration between the start and end date:
| Unit | Description |
|---|---|
hour | Limits reset every hour |
day | Limits reset every day |
year | Limits reset every year |
Budgets per consumer
One budget can give each of your consumers their own limit. Set partition_by to an expression of the
Otoroshi expression language: the budget
counts apart every value the expression takes for a call, and each value has the whole limit for itself. A team
of 200 people gets "200 $ per person per month" with a single budget, and everyone keeps their own counter.
{
"limits": { "total_usd": 200 },
"duration": { "value": 30, "unit": "day" },
"scope": {
"always_apply_rules": true,
"rules": [ { "path": "$.provider.metadata.ai_studio_workspace", "value": "v4e46mctfxxw" } ],
"partition_by": "${user.email || apikey.metadata.ai_studio_owner}"
}
}
Pick what a consumer is with the expression:
| Expression | Each consumer is |
|---|---|
${user.email || apikey.metadata.ai_studio_owner} | a person: their chats and the calls of the API keys they own share one limit |
${consumer.id} | each user, and each API key |
${user.email || apikey.metadata.ai_studio_owner || apikey.clientId} | a person, and each API key that has no owner |
${apikey.metadata.team} | a team, from the metadata of its API keys |
${user.email || apikey.metadata.ai_studio_owner}/${ctx.ai.model} | a person, for each model |
Besides everything the expression language reads (the user, the API key and their metadata, the route), the
provider and the model a call counts for are available as ${ctx.ai.provider.id}, ${ctx.ai.provider.name} and
${ctx.ai.model}.
A call the expression finds nothing for, like a call made with an API key that has no owner, is not counted by
that budget: it stays under the other budgets of its scope. To count those calls together, give the expression a
default value, ${apikey.metadata.team:none}, or ${user.email || apikey.metadata.ai_studio_owner :: shared} for
a chain. Values are compared in lower case, so Jane@Acme.io and jane@acme.io share their counter.
Count consumers by what they cannot choose: their identity, their API key, their route. A value the caller sets freely, like a request header, would let anyone start over with a whole new limit.
In a cluster, every node counts the consumers of a budget apart once it runs a version that knows partition_by:
upgrade your nodes, then create these budgets.
The consumption of a budget counted per consumer is the one of a consumer: name them with ?partition= on the
consumption endpoints, and list them with the admin API. Alerts carry the consumer too:
AiBudgetAlmostExceeded has a partition field, AiBudgetExceeded a partitions object with the consumer of
each budget it names.
Action on exceed
A budget can block requests on exceeded limits or just emit alerts.

| Parameter | Type | Default | Description |
|---|---|---|---|
mode | string | — | "soft" (alerts only) or "block" (deny requests with HTTP 429) |
alert_on_exceed | boolean | — | Emit an alert when the budget is exceeded |
alert_on_almost_exceed | boolean | — | Emit an alert when the budget is almost exceeded |
alert_on_almost_exceed_percentage | number | 80 | Percentage threshold for "almost exceeded" alerts (e.g. 80 means alert at 80% consumption) |
error_message | string | budget exceeded | Error returned to the client of a call blocked by the budget |
A blocked call tells your users what happened in your own words, for instance who to contact to get more budget.
The response always carries budget_exceeded, so clients, metrics and dashboards recognize it whatever the message:
{
"error": "Your AI budget is exhausted, please contact your administrator",
"budget_exceeded": true
}
When several budgets block the same call, the message of one of them is returned. Budgets in soft mode never
block a call, so their message is never returned.
Extensions
Someone used their whole budget and needs to finish what they started? Let them take an extension. The administrator of a budget decides what an extension adds and how many can be taken in a cycle; the person the budget counts takes one when they need it, in AI Studio or through the admin API, without waiting for anyone.
"extensions": {
"enabled": true,
"max_per_cycle": 1,
"limits": { "total_usd": 10 }
}
With this budget of 200 $ per person and per month, Michel, who spent his 200 $, takes his extension of the month: he can go on up to 210 $, and the budget blocks him again past it. His consumption stays the one he really had, 200 $ out of 210 $, in every report.
| Parameter | Type | Default | Description |
|---|---|---|---|
enabled | boolean | false | Whether extensions can be taken |
max_per_cycle | number | 1 | How many extensions can be taken in a cycle, by each consumer for a budget per consumer |
limits | object | — | What one extension adds, by limit (total_usd, total_tokens, inference_usd…): only the limits of the budget |
- An extension counts for the current cycle only: the next one starts with the limit of the budget.
- Extensions add up, up to
max_per_cycle. Two taken at the very same time never go past it. - On a budget per consumer, each person takes their own, once they consumed in the cycle. On a budget counted as a whole, the extension raises a limit everyone shares, so it is taken by an administrator.
- An administrator can cancel the extensions of a cycle: their amounts are removed, and they are not given back. A reset starts the cycle over, extensions included.
- Every extension taken or cancelled is audited (
AiBudgetExtended,AiBudgetExtensionCancelled), with who it was for and who took it.
The consumption of a budget then tells what was extended: allowed_total_usd includes the extensions,
extra_total_usd is what they added, extensions_taken and extensions_max how many were and can be taken, and
extension what the next one would add.
Admin API
in addition to the classic admin API endpoints for the ai-gateway.extensions.cloud-apim.com/AiBudget entity, you have
- GET /api/extensions/cloud-apim/extensions/ai-extension/budgets/:id/consumption
- POST /api/extensions/cloud-apim/extensions/ai-extension/budgets/:id/consumption/_reset
- GET /api/extensions/cloud-apim/extensions/ai-extension/budgets/:id/partitions
- POST /api/extensions/cloud-apim/extensions/ai-extension/budgets/:id/consumption/_extend
- DELETE /api/extensions/cloud-apim/extensions/ai-extension/budgets/:id/consumption/_extend
For a budget per consumer, name the consumer with ?partition=jane@acme.io to read their
consumption, reset it, take or cancel their extensions. Without it, the reset starts the cycle over for everyone.
/partitions lists the consumers of the current cycle, the ones that consumed most first, with limit, offset
and a q filter:
{
"total": 2,
"items": [
{ "partition": "jane@acme.io", "consumption": { "consumed_total_usd": 182.4, "allowed_total_usd": 210, "extra_total_usd": 10 } },
{ "partition": "john@acme.io", "consumption": { "consumed_total_usd": 37.9, "allowed_total_usd": 200 } }
]
}
_extend takes one of the extensions the budget allows, with no body, and returns the consumption
it leads to; DELETE cancels the extensions of the cycle.
the consumption endpoint returns a response like:
{
"consumed_total_usd": 0.00188655,
"consumed_total_tokens": 8036,
"consumed_inference_usd": 0.00188655,
"consumed_inference_tokens": 8036,
"consumed_image_usd": 0,
"consumed_image_tokens": 0,
"consumed_audio_usd": 0,
"consumed_audio_tokens": 0,
"consumed_video_usd": 0,
"consumed_video_tokens": 0,
"consumed_embedding_usd": 0,
"consumed_embedding_tokens": 0,
"consumed_moderation_usd": 0,
"consumed_moderation_tokens": 0,
"consumed_decision_usd": 0,
"consumed_decision_tokens": 0,
"remaining_total_usd": 199.99811345,
"remaining_total_tokens": 9991964,
"remaining_inference_usd": 0,
"remaining_inference_tokens": 0,
"remaining_image_usd": 0,
"remaining_image_tokens": 0,
"remaining_audio_usd": 0,
"remaining_audio_tokens": 0,
"remaining_video_usd": 0,
"remaining_video_tokens": 0,
"remaining_embedding_usd": 0,
"remaining_embedding_tokens": 0,
"remaining_moderation_usd": 0,
"remaining_moderation_tokens": 0,
"remaining_decision_usd": 0,
"remaining_decision_tokens": 0,
"allowed_total_usd": 200,
"allowed_total_tokens": 10000000,
"allowed_inference_usd": null,
"allowed_inference_tokens": null,
"allowed_image_usd": null,
"allowed_image_tokens": null,
"allowed_audio_usd": null,
"allowed_audio_tokens": null,
"allowed_video_usd": null,
"allowed_video_tokens": null,
"allowed_embedding_usd": null,
"allowed_embedding_tokens": null,
"allowed_moderation_usd": null,
"allowed_moderation_tokens": null,
"allowed_decision_usd": null,
"allowed_decision_tokens": null
}
Embed in responses
The budget embedded in a response is the one of the caller: on a budget per consumer,
partition names them, and the extra_* and extensions_* fields tell the extensions they took.
curl --request POST \
--url 'http://proxy.oto.tools:8080/v1/chat/completions?embed_budget=true' \
--header 'content-type: application/json' \
--data '{
"model": "anthropic/claude-sonnet-4-5-20250929",
"stream": false,
"messages": [
{
"role": "user",
"content": "tell me a joke ?"
}
]
}'
{
"id": "chatcmpl-9ODWHjtFVS5qzpU1nc79TprfRhZElq0e",
"object": "chat.completion",
"created": 1762441013,
"model": "claude-sonnet-4-5-20250929",
"system_fingerprint": "fp-tEQXbmpSPELEh3fYd5Rs9Jf4HIaI3E6s",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Why don't scientists trust atoms?\n\nBecause they make up everything! 😄"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 20,
"total_tokens": 32,
"completion_tokens_details": {
"reasoning_tokens": -1
}
},
"budget": {
"consumed_total_usd": 0.00394755,
"consumed_total_tokens": 8231,
"consumed_inference_usd": 0.00394755,
"consumed_inference_tokens": 8231,
"consumed_image_usd": 0,
"consumed_image_tokens": 0,
"consumed_audio_usd": 0,
"consumed_audio_tokens": 0,
"consumed_video_usd": 0,
"consumed_video_tokens": 0,
"consumed_embedding_usd": 0,
"consumed_embedding_tokens": 0,
"consumed_moderation_usd": 0,
"consumed_moderation_tokens": 0,
"consumed_decision_usd": 0,
"consumed_decision_tokens": 0,
"remaining_total_usd": 199.99605245,
"remaining_total_tokens": 9991769,
"remaining_inference_usd": 0,
"remaining_inference_tokens": 0,
"remaining_image_usd": 0,
"remaining_image_tokens": 0,
"remaining_audio_usd": 0,
"remaining_audio_tokens": 0,
"remaining_video_usd": 0,
"remaining_video_tokens": 0,
"remaining_embedding_usd": 0,
"remaining_embedding_tokens": 0,
"remaining_moderation_usd": 0,
"remaining_moderation_tokens": 0,
"remaining_decision_usd": 0,
"remaining_decision_tokens": 0,
"allowed_total_usd": 200,
"allowed_total_tokens": 10000000,
"allowed_inference_usd": null,
"allowed_inference_tokens": null,
"allowed_image_usd": null,
"allowed_image_tokens": null,
"allowed_audio_usd": null,
"allowed_audio_tokens": null,
"allowed_video_usd": null,
"allowed_video_tokens": null,
"allowed_embedding_usd": null,
"allowed_embedding_tokens": null,
"allowed_moderation_usd": null,
"allowed_moderation_tokens": null,
"allowed_decision_usd": null,
"allowed_decision_tokens": null
}
}