Reporting

You can use the audit events generated by your LLM usage to make some reporting dashboard and follow your metrics in near real-time.

Every interaction with an LLM provider generates a LLMUsageAudit event that contains detailed information about the request, the response, the provider, the costs, the ecological impacts, the budgets, and more.

LLMUsageAudit event fields

Field	Type	Description
`@id`	string	Unique event identifier
`@timestamp`	number	Event timestamp in milliseconds
`@type`	string	Always `"AuditEvent"`
`@product`	string	Always `"otoroshi"`
`audit`	string	Always `"LLMUsageAudit"`
`provider_kind`	string	The LLM provider type (e.g., `openai`, `anthropic`, `mistral`, etc.)
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
`model`	string	The model used for the request
`rate_limit`	object	Rate limit information from the provider (`requests_limit`, `requests_remaining`, `tokens_limit`, `tokens_remaining`)
`usage`	object	Token usage (`prompt_tokens`, `generation_tokens`, `reasoning_tokens`)
`error`	object/null	Error details if the request failed, `null` on success
`consumed_using`	string	The type of operation (see below)
`user`	object/null	The authenticated user, if any
`apikey`	object/null	The API key used, if any
`route`	object/null	The Otoroshi route that handled the request
`input_prompt`	array/object	The input prompt or request body
`output`	object/array	The LLM response
`provider_details`	object	Full provider configuration
`impacts`	object/null	Ecological impact data (if enabled)
`costs`	object/null	Cost tracking data (if enabled)
`budgets`	object/null	Budget consumption data (if enabled)
`consumer_rate_limit`	object/null	Consumer token rate limit information

Operation types (`consumed_using`)

The consumed_using field indicates what kind of LLM operation generated the event:

Value	Description
`chat/completion/blocking`	Synchronous chat completion
`chat/completion/streaming`	Streaming chat completion
`completion/blocking`	Synchronous text completion
`completion/streaming`	Streaming text completion
`embedding_model/embedding`	Text embedding generation
`audio_model/translate`	Audio translation
`audio_model/stt`	Speech-to-text transcription
`image_model/generate`	Image generation
`image_model/edit`	Image editing
`moderation_model/moderate`	Content moderation
`video_model/generate`	Video generation

Dashboard example

In the following screenshot, we used an Elasticsearch data exporter to send LLMUsageAudit events to an Elasticsearch instance and Kibana to create a dashboard based on those events.

Elastic data exporter example

Go to Data Exporters page and create a new one.

{
  "_loc": {
    "tenant": "default",
    "teams": [
      "default"
    ]
  },
  "type": "elastic",
  "enabled": true,
  "id": "data_exporter_be7a6e21-152b-4c09-b384-15f9dbb8041f",
  "name": "New elastic exporter config",
  "desc": "New elastic exporter config",
  "metadata": {},
  "tags": [],
  "bufferSize": 5000,
  "jsonWorkers": 1,
  "sendWorkers": 5,
  "groupSize": 100,
  "groupDuration": 30000,
  "projection": {},
  "filtering": {
    "include": [
      {
        "audit": "LLMUsageAudit"
      }
    ],
    "exclude": []
  },
  "config": {
    "clusterUri": "http://localhost:9200",
    "uris": [
      "http://localhost:9200"
    ],
    "index": "otoroshi-llm",
    "type": null,
    "user": null,
    "password": null,
    "headers": {},
    "indexSettings": {
      "clientSide": true,
      "interval": "Day",
      "numberOfShards": 1,
      "numberOfReplicas": 1
    },
    "mtlsConfig": {
      "certs": [],
      "trustedCerts": [],
      "mtls": false,
      "loose": false,
      "trustAll": false
    },
    "applyTemplate": true,
    "version": null,
    "maxBulkSize": 100,
    "sendWorkers": 4
  },
  "kind": "events.otoroshi.io/DataExporter"
}

Extra analytics data in GatewayEvent

In addition to the dedicated LLMUsageAudit events, the LLM extension also enriches the standard Otoroshi GatewayEvent (the analytics event generated for every HTTP request passing through a route) with LLM-specific data. This data is added to the ExtraAnalyticsData field of the GatewayEvent using different keys depending on the operation type.

Analytics data keys

Key	Operation type	Added by
`ai`	Chat completion, text completion	LLM providers (OpenAI, Anthropic, Mistral, etc.)
`ai-embedding`	Embedding generation	Embedding model auditing decorator
`ai-audio`	Audio translation, speech-to-text	Audio model auditing decorator
`ai-image`	Image generation, image editing	Image model auditing decorator
`ai-moderation`	Content moderation	Moderation model auditing decorator
`ai-video`	Video generation	Video model auditing decorator
`ai-consumer-rate-limit`	Token rate limiting	LLM Token Rate Limiting plugin

Each key contains an array of objects, one per LLM call made during the request processing. This allows tracking multiple LLM calls within a single HTTP request (e.g., when using fallbacks or load balancing).

`ai` key structure (chat/completion)

For chat and text completion operations, each provider adds a slug to the ai array with the following fields:

Field	Type	Description
`provider_kind`	string	The provider type (e.g., `"openai"`, `"AzureOpenAi"`, `"Anthropic"`)
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
`model`	string	The model used
`rate_limit`	object	Rate limit information (`requests_limit`, `requests_remaining`, `tokens_limit`, `tokens_remaining`)
`usage`	object	Token usage (`prompt_tokens`, `generation_tokens`, `reasoning_tokens`)
`cache`	object	Cache information (if a cache hit occurred)
`deployment_id`	string	Azure OpenAI deployment ID (Azure providers only)
`resource_name`	string	Azure OpenAI resource name (Azure providers only)

`ai-consumer-rate-limit` key structure

The LLM Token Rate Limiting plugin adds an ai-consumer-rate-limit array to the extra analytics data. Each entry in the array is appended during response or error transformation, with the following fields:

Field	Type	Description
`max_tokens`	number	Maximum allowed tokens in the window
`window_millis`	number	Rate limit window duration in milliseconds
`consumed_tokens`	number	Tokens consumed in the current window
`remaining_tokens`	number	Remaining tokens in the current window

`ai-embedding` key structure

Field	Type	Description
`provider_kind`	string	The provider type
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
+ response fields		The full embedding response (model, usage, data)

`ai-audio` key structure

Field	Type	Description
`provider_kind`	string	The provider type
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
+ response fields		The full audio response (text, language, duration, segments)

`ai-image` key structure

Field	Type	Description
`provider_kind`	string	The provider type
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
+ response fields		The full image response (created, data)

`ai-moderation` key structure

Field	Type	Description
`provider_kind`	string	The provider type
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
+ response fields		The full moderation response (id, model, results)

`ai-video` key structure

Field	Type	Description
`provider_kind`	string	The provider type
`provider`	string	The provider entity ID
`duration`	number	Request duration in milliseconds
+ response fields		The full video response

Example GatewayEvent extra analytics data

{
  "ai": [
    {
      "provider_kind": "openai",
      "provider": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
      "duration": 415,
      "model": "gpt-4o-mini",
      "rate_limit": {
        "requests_limit": 10000,
        "requests_remaining": 9999,
        "tokens_limit": 200000,
        "tokens_remaining": 199993
      },
      "usage": {
        "prompt_tokens": 11,
        "generation_tokens": 18,
        "reasoning_tokens": 0
      }
    }
  ],
  "ai-consumer-rate-limit": [
    {
      "max_tokens": 1000,
      "window_millis": 60000,
      "consumed_tokens": 29,
      "remaining_tokens": 971
    }
  ]
}

LLMUsageAudit event example

Here is a full example of a LLMUsageAudit event for a chat completion request:

{
  "@id": "1905616593920983819",
  "@timestamp": 1743169375292,
  "@type": "AuditEvent",
  "@product": "otoroshi",
  "@serviceId": "",
  "@service": "Otoroshi",
  "@env": "dev",
  "audit": "LLMUsageAudit",
  "provider_kind": "openai",
  "provider": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
  "duration": 415,
  "model": "gpt-4o-mini",
  "rate_limit": {
    "requests_limit": 10000,
    "requests_remaining": 9999,
    "tokens_limit": 200000,
    "tokens_remaining": 199993
  },
  "usage": {
    "prompt_tokens": 11,
    "generation_tokens": 18,
    "reasoning_tokens": 0
  },
  "error": null,
  "consumed_using": "chat/completion/blocking",
  "user": null,
  "apikey": null,
  "route": {
    "id": "route_e4a9d6cb3-d859-4203-a860-8d1dd6d09557",
    "name": "test",
    "..."
  },
  "input_prompt": [
    {
      "role": "user",
      "content": "tell me a joke"
    }
  ],
  "output": {
    "generations": [
      {
        "message": {
          "role": "assistant",
          "content": "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
        }
      }
    ],
    "metadata": {
      "rate_limit": { "..." },
      "usage": { "..." },
      "costs": {
        "input_cost": 0.00000165,
        "output_cost": 0.0000108,
        "reasoning_cost": 0,
        "total_cost": 0.00001245,
        "currency": "dollar"
      }
    }
  },
  "provider_details": {
    "id": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
    "name": "OpenAI",
    "provider": "openai",
    "..."
  },
  "impacts": null,
  "costs": {
    "input_cost": 0.00000165,
    "output_cost": 0.0000108,
    "reasoning_cost": 0,
    "total_cost": 0.00001245,
    "currency": "dollar"
  },
  "budgets": null,
  "consumer_rate_limit": null
}

Reporting

LLMUsageAudit event fields​

Operation types (consumed_using)​

Dashboard example​

Elastic data exporter example​

Extra analytics data in GatewayEvent​

Analytics data keys​

ai key structure (chat/completion)​

ai-consumer-rate-limit key structure​

ai-embedding key structure​

ai-audio key structure​

ai-image key structure​

ai-moderation key structure​

ai-video key structure​

Example GatewayEvent extra analytics data​

LLMUsageAudit event example​