Skip to main content

OpenAI Compatible API (unified)

The LLM OpenAI Compatible API plugin is a unified backend plugin that exposes a single Otoroshi route as a full-featured, multi-endpoint AI API. Instead of configuring separate plugins for chat completions, audio, images, embeddings, moderation, and responses, this single plugin handles all of them with a consistent configuration.

cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.OpenAiCompatApi

Why use this plugin?​

When building an AI gateway, you typically need to expose several API endpoints: chat completions, embeddings, image generation, audio transcription, moderation, etc. With the individual plugins, you would need to configure each one separately on the same route, mapping each to a specific path.

The OpenAI Compatible API plugin simplifies this by providing a single plugin that routes requests based on the URL path to the appropriate handler. It also goes beyond the standard OpenAI API by supporting additional endpoints:

  • Anthropic Messages API (/messages) for clients like Claude Code
  • OpenAI Responses API (/responses) for the newer responses format
  • Open Responses (/open-responses) for the Open Responses specification
  • Prompt contexts (/contexts) for listing available prompt contexts

This makes it the ideal choice when you want to expose a single, coherent API surface that supports multiple AI client ecosystems.

Supported endpoints​

MethodPathDescriptionConfig field
GET/modelsList available models from all configured providerslanguage_model_refs
GET/providersList every provider type the gateway can talk to, with their capabilities—
GET/model-capabilitiesList the model types (modalities) the gateway supports—
GET/contextsList available prompt contextscontext_refs
POST/chat/completionsOpenAI Chat Completions APIlanguage_model_refs
POST/responsesOpenAI Responses API (or Open Responses if configured)language_model_refs
POST/open-responsesOpen Responses API (always available)language_model_refs
POST/oai-responsesOpenAI Responses API (always available)language_model_refs
POST/messagesAnthropic Messages APIlanguage_model_refs
POST/embeddingsOpenAI Embeddings APIembedding_model_refs
POST/images/generationsOpenAI Image Generation APIimage_model_refs
POST/images/editsOpenAI Image Edit APIimage_model_refs
POST/audio/speechOpenAI Text-to-Speech APIaudio_model_refs
POST/audio/transcriptionsOpenAI Speech-to-Text APIaudio_model_refs
POST/audio/translationsOpenAI Audio Translation APIaudio_model_refs
POST/moderationsOpenAI Moderation APImoderation_model_refs
POST/ocrOCR (text extraction from images and pdf)ocr_model_refs
POST/mcpMCP Streamable HTTP endpoint serving an MCP virtual servermcp_server_ref

Plugin configuration​

{
  "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.OpenAiCompatApi",
  "enabled": true,
  "config": {
    "language_model_refs": ["provider_openai_1", "provider_mistral_1"],
    "audio_model_refs": ["audio_model_1"],
    "image_model_refs": ["image_model_1"],
    "ocr_model_refs": ["ocr_model_1"],
    "embedding_model_refs": ["embedding_model_1"],
    "moderation_model_refs": ["moderation_model_1"],
    "context_refs": ["context_1", "context_2"],
    "max_size_upload": 104857600,
    "decode_images": false,
    "use_open_response_for_responses": false
  }
}
ParameterTypeDefaultDescription
language_model_refsarray of strings[]References to LLM provider entities used for chat completions, responses, and messages endpoints
audio_model_refsarray of strings[]References to audio model entities used for speech and transcription endpoints
image_model_refsarray of strings[]References to image model entities used for image generation and editing
embedding_model_refsarray of strings[]References to embedding model entities used for the embeddings endpoint
moderation_model_refsarray of strings[]References to moderation model entities used for the moderations endpoint
ocr_model_refsarray of strings[]References to OCR model entities used for the /ocr endpoint
context_refsarray of strings[]References to prompt context entities returned by the /contexts endpoint
max_size_uploadnumber104857600 (100MB)Maximum file upload size in bytes for audio, image, and OCR endpoints
decode_imagesbooleanfalseWhen enabled, decodes base64-encoded image results into binary responses
use_open_response_for_responsesbooleanfalseWhen enabled, the /responses endpoint uses the Open Responses proxy instead of the default OpenAI Responses proxy
mcp_server_refstringnullReference to the MCP virtual server served on /mcp. Without it (or when the server is disabled), /mcp answers 404

Route configuration example​

{
  "id": "route_unified_ai_api",
  "name": "Unified AI API",
  "frontend": {
    "domains": ["api.domain.tld/v1"],
    "strip_path": true,
    "exact": false
  },
  "backend": {
    "targets": [
      {
        "id": "target_1",
        "hostname": "request.otoroshi.io",
        "port": 443,
        "tls": true
      }
    ]
  },
  "plugins": [
    {
      "enabled": true,
      "plugin": "cp:otoroshi.next.plugins.OverrideHost",
      "config": {}
    },
    {
      "enabled": true,
      "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.OpenAiCompatApi",
      "config": {
        "language_model_refs": ["provider_openai_1", "provider_mistral_1"],
        "embedding_model_refs": ["embedding_model_1"],
        "image_model_refs": ["image_model_1"],
        "audio_model_refs": ["audio_model_1"],
        "context_refs": ["context_project_a"],
        "use_open_response_for_responses": false
      }
    }
  ]
}

Endpoint details​

GET /models​

Lists all models available from the configured language model providers. Compatible with the OpenAI Models API.

curl https://api.domain.tld/v1/models \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

When multiple providers are configured, model IDs are prefixed with the provider slug name (e.g., my_openai/gpt-4o, or my_openrouter###openai/gpt-4o when the model id already contains a /).

Query parameters​

ParameterValuesDescription
enrichedtrue, 1 or no valueAdds a _metadata object to each model: types, cost, API, capabilities, modalities, limits and prices. See enriched models
kind (or kinds)text, audio, image, ocr, embedding, moderation, videoOnly lists the models of at least one of these types. See filter models by type
endpoint (or endpoints)chat_completions, completions, responses, embeddings, images_generations, images_edits, audio_speech, audio_transcriptions, audio_translations, moderations, ocr, realtime, videosOnly lists the models served on at least one of these OpenAI API endpoints. See filter models by endpoint
has_costtrue, 1 or no value / false, 0Only lists the models cost tracking can (or cannot) put a price on. See filter models by cost
rawtrueAsks the provider for the model ids of its API rather than its model names, for the providers making the difference (OVH AI Endpoints, when not unified)

kind and endpoint accept several values, by repeating the parameter or with a comma separated list, and values are case insensitive. Filters combine: a model is listed when it matches all of them. They never add the _metadata object by themselves, add enriched=true for that.

curl "https://api.domain.tld/v1/models?kind=text&endpoint=chat_completions&has_cost=true&enriched=true" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

Enriched models​

Add ?enriched=true to get everything you need to build a model picker, route by capability or show prices in your own UI: each model comes with a _metadata object describing what it can do, how much it can take and what it costs.

curl "https://api.domain.tld/v1/models?enriched=true" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

Response:

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4o",
      "object": "model",
      "created": 1789575515,
      "owned_by": "OpenAI",
      // ...
      "_metadata": {
        "kinds": ["text"],
        "has_cost": true,
        "openai_compatible": true,
        "endpoints": ["chat_completions"],
        "mode": "chat",
        "capabilities": {
          "reasoning": false,
          "tool_call": true,
          "structured_output": true,
          "temperature": true,
          "attachment": true,
          "prompt_caching": true
        },
        "modalities": {
          "input": ["text", "image", "pdf"],
          "output": ["text"]
        },
        "limits": {
          "context": 128000,
          "output": 16384
        },
        "pricing": {
          "prompt": "0.0000025",
          "completion": "0.00001",
          "input_cache_read": "0.00000125"
        },
        "knowledge": "2023-09",
        "sources": {
          "catalog": { "provider": "openai", "model": "gpt-4o", "match": "exact" },
          "pricing": { "source": "price-table", "model": "gpt-4o" }
        }
      }
    }
  ]
}

The metadata combines the models.dev catalog, bundled with the extension, with the price table used by cost tracking:

FieldDescription
kindsThe model types the model belongs to: text, audio, image, ocr, embedding, moderation, video
has_costWhether cost tracking puts a cost on the calls to this model, so they count against your dollar budgets
openai_compatibleWhether the model is served through an OpenAI compatible API: the gateway talks to its provider in the OpenAI format (OpenAI, Azure OpenAI, Groq, xAI, Gemini, OVHcloud, OpenRouter and every OpenAI compatible provider), the provider serves this very model that way, and on at least one OpenAI endpoint. Models of Anthropic, Mistral, Cohere or Ollama are false: the gateway translates for them. Left out for routers, whose models are served by other providers
endpointsThe OpenAI API endpoints the provider serves the model on, when it is openai_compatible (see below)
modeWhat the price table bills the model as (chat, responses, embedding, ...), when it knows it
capabilities.reasoning, capabilities.reasoning_optionsWhether the model thinks before answering, and the reasoning controls it accepts (effort levels, token budget)
capabilities.tool_call, capabilities.structured_outputTool calling and JSON schema outputs support
capabilities.temperature, capabilities.attachmentTemperature control and file attachments support
capabilities.prompt_caching, capabilities.web_searchPrompt caching and built-in web search support
modalities.input, modalities.outputAccepted and produced content: text, image, pdf, audio, video
limits.context, limits.input, limits.outputContext window, maximum input and maximum output, in tokens
pricingThe prices cost tracking bills the model with, in dollars, as strings, in the OpenRouter format: per token for prompt, completion, input_cache_read, input_cache_write, internal_reasoning, audio, audio_output, per image for image, image_output. Models billed in another unit get input_second and output_second (per second of audio), input_character (per character of text to speak) or video_second (per second of generated video) instead of token prices
knowledgeKnowledge cutoff
status, deprecation_datebeta or deprecated status, and the announced deprecation date
sourcesWhere the metadata comes from (see below)

Fields the sources know nothing about are left out: a model nobody knows only comes with its kinds, guessed from its name, and has_cost.

The prices are exactly the ones your budgets are charged with, in dollars: sources.pricing.source is price-table when they come from the price table (including your own custom-prices), and models.dev when the catalog fills the gap. Prices published in another currency, like the euros of Scaleway and OVHcloud, are converted, and sources.pricing gives their currency and the exchange_rate applied.

Providers rarely spell a model id the same way, so each listed model is matched against what its own provider serves first, then against the rest of the catalog. sources.catalog.match tells how the model was recognized:

MatchMeaningPricing
exactThe very id the provider serves (models/gemini-2.5-flash is gemini-2.5-flash)✅
normalizedThe same model once cosmetic differences are removed: case, dated snapshots, . versus -, routing variants like :free✅
approximateThe same model family for this provider, a short version suffix like -2411 or -001 aside❌
global, global-approximateThe same model served by another provider, which is perfect for capabilities and limits, but not for prices❌

Filter models by type​

Add ?kind= to only list the models of one of the model types Otoroshi LLM supports: text, audio, image, ocr, embedding, moderation or video. It is the quickest way to find the model to put in an embedding model, an audio model or an image model entity, among everything a provider serves.

curl "https://api.domain.tld/v1/models?kind=embedding"   -H "Authorization: Bearer $OTOROSHI_BEARER"

Ask for several types by repeating the parameter or with a comma separated list (?kind=image&kind=video, ?kind=image,video): a model is listed as soon as it belongs to one of them. The filter combines with ?enriched=true.

A model can belong to several types, an omni model both chats and speaks for instance. The types are worked out from what the model is for, then from what it produces. A model only served on image or realtime endpoints is not a text model, even when it answers with some text:

TypeModels
embeddingEmbedding models (text-embedding-3-small, mistral-embed, e5-large-v2...)
moderationModeration and safety models (omni-moderation-latest, llama-guard-4-12b, gpt-oss-safeguard-20b...)
ocrDocument text extraction models (mistral-ocr-latest, deepseek-ocr, paddleocr-vl...), and every model of an OCR provider like AlphaEdge
audioModels producing audio (tts-1, gpt-4o-mini-tts, omni models) and speech recognition models (whisper-1, gpt-4o-transcribe, qwen3-asr-flash)
imageModels producing images (gpt-image-1, gemini-2.5-flash-image, flux...)
videoModels producing videos (sora-2, veo-3...)
textModels producing text, apart from the ones above: a chat model that understands audio or images stays a text model

Rerankers belong to none of these types, so they only show up when no filter is set.

Filter models by endpoint​

Add ?endpoint= to only list the models their provider serves on one of these OpenAI API endpoints: chat_completions, completions, responses, embeddings, images_generations, images_edits, audio_speech, audio_transcriptions, audio_translations, moderations, ocr, realtime or videos. It keeps out of a chat the models that would fail there, like gpt-5-pro, only served on responses, or the realtime models.

curl "https://api.domain.tld/v1/models?endpoint=chat_completions" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

The endpoints come from the price table when it lists them (gpt-5.5 is served on chat_completions and responses, gpt-image-1 on images_generations and images_edits), from what the price table bills the model as otherwise, and from the model types as a last resort. They are only given for the openai_compatible models, so the filter only keeps those. Repeat the parameter or use a comma separated list to accept several endpoints.

Filter models by cost​

Add ?has_cost=true to only list the models cost tracking can put a price on, and ?has_cost=false for the others. Perfect to offer only the models your dollar budgets can keep in check, or to spot the ones still missing a custom price.

curl "https://api.domain.tld/v1/models?kind=text&has_cost=true" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

A model has a cost exactly when a call to it would be billed: the provider is priced by cost tracking, and either its price is known (from your custom prices, the price table or the models.dev catalog, honouring the costs-tracking-provider and costs-tracking-model metadata of the provider) or the provider reports the cost of each call itself, like OpenRouter.

?enriched=true, ?kind=, ?endpoint= and ?has_cost= combine, and also work on the models list plugins.

GET /contexts​

Returns the list of prompt contexts configured for this plugin. Each context includes its id and name.

curl https://api.domain.tld/v1/contexts \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

Response:

[
  { "id": "context_project_a", "name": "Project A context" },
  { "id": "context_support", "name": "Support context" }
]

GET /providers​

Returns the catalog of every provider the gateway can connect to, each with the capabilities (modalities) it supports. This is a great way to build a provider picker in your own UI, or to discover programmatically which providers can handle a given task — text, audio, image, OCR, embeddings, moderation or video.

curl https://api.domain.tld/v1/providers \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

Response:

{
  "object": "list",
  "data": [
    { "id": "openai", "label": "OpenAI", "capabilities": ["text", "audio", "image", "embedding", "moderation"] },
    { "id": "anthropic", "label": "Anthropic", "capabilities": ["text"] },
    { "id": "mistral", "label": "Mistral", "capabilities": ["text", "audio", "ocr", "embedding", "moderation"] },
    { "id": "ovh-ai-endpoints", "label": "OVH AI Endpoints", "capabilities": ["text", "audio", "image", "embedding", "moderation"] }
  ]
}

Filtering by capability​

Pass one or more capabilities query parameters to keep only the providers that expose all of them. The parameter is repeatable and also accepts a comma-separated list, so the two calls below are equivalent:

curl "https://api.domain.tld/v1/providers?capabilities=image&capabilities=text" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

curl "https://api.domain.tld/v1/providers?capabilities=image,text" \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

This returns only the providers that can do both image and text.

The same catalog is also available as a standalone plugin you can put on its own route:

cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.LlmProvidersCatalog

GET /model-capabilities​

Returns the model types (modalities) the gateway supports — text, audio, image, ocr, embedding, moderation and video — each with the list of providers that expose it. It is the mirror image of /providers: where /providers answers "what can this provider do?", /model-capabilities answers "which providers can do this?".

curl https://api.domain.tld/v1/model-capabilities \
  -H "Authorization: Bearer $OTOROSHI_BEARER"

Response:

{
  "object": "list",
  "data": [
    { "id": "text", "label": "Text", "providers": ["anthropic", "openai", "mistral", "..."] },
    { "id": "audio", "label": "Audio", "providers": ["openai", "groq", "elevenlabs", "mistral", "ovh-ai-endpoints", "..."] },
    { "id": "image", "label": "Image", "providers": ["openai", "gemini", "luma", "ovh-ai-endpoints", "..."] },
    { "id": "ocr", "label": "OCR", "providers": ["alphaedge", "mistral"] },
    { "id": "embedding", "label": "Embedding", "providers": ["openai", "mistral", "cohere", "ovh-ai-endpoints", "..."] },
    { "id": "moderation", "label": "Moderation", "providers": ["openai", "mistral", "ovh-ai-endpoints"] },
    { "id": "video", "label": "Video", "providers": ["luma", "openrouter"] }
  ]
}

The same list is also available as a standalone plugin you can put on its own route:

cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.LlmModelCapabilities
Also available on the Otoroshi admin API

Both catalogs are exposed on the Otoroshi admin API as well (authenticated with an admin API key), so you can query them without exposing a route:

  • GET /api/extensions/cloud-apim/extensions/ai-extension/providers
  • GET /api/extensions/cloud-apim/extensions/ai-extension/model-capabilities
curl "https://otoroshi-api.domain.tld/api/extensions/cloud-apim/extensions/ai-extension/providers?capabilities=image,text" \
  -u "$ADMIN_APIKEY_ID:$ADMIN_APIKEY_SECRET"

The capabilities filter behaves exactly like on /providers above, and the response shape is identical.

POST /chat/completions​

Standard OpenAI Chat Completions API endpoint. Supports streaming, tool calling, and model routing.

curl https://api.domain.tld/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      { "role": "user", "content": "Hello!" }
    ]
  }'

Multipart content​

Messages can mix text with images, audio and files. Documents are attached with a file content part, either inline as a data-uri or by url, and are forwarded to the target provider in its own format:

{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Summarize this document" },
        { "type": "file", "file": {
          "filename": "report.pdf",
          "file_data": "data:application/pdf;base64,JVBERi0xLjQK..."
        }}
      ]
    }
  ]
}

PDF and plain text documents are supported. The same document is sent as a file part to OpenAI compatible providers and as a document block to Anthropic, so a single request works across providers.

Model routing​

When multiple providers are configured, you can target a specific provider using the model field:

  • Slash syntax: providerName/modelName (e.g., my_openai/gpt-4o)
  • Hash syntax: providerId###modelName (e.g., provider_xxx###gpt-4o)

If no provider prefix is specified, the first configured provider is used.

Streaming​

Streaming is activated when any of the following is true:

  • The request body contains "stream": true
  • The query parameter ?stream=true is present
  • The header x-stream: true is present

POST /responses​

OpenAI Responses API endpoint. This is the newer OpenAI API format that uses input and instructions instead of messages.

curl https://api.domain.tld/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "gpt-4o",
    "instructions": "You are a helpful assistant.",
    "input": "What is the capital of France?"
  }'

Response:

{
  "id": "resp_xxxxx",
  "object": "response",
  "created_at": 1711569952,
  "model": "gpt-4o",
  "status": "completed",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of France is Paris.",
          "annotations": []
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 25,
    "output_tokens": 8,
    "total_tokens": 33
  }
}

Input format​

The input field accepts multiple formats:

  • String: a simple text message treated as a user message

    { "input": "Hello!" }

  • Array of messages: standard role-based messages

    {
      "input": [
        { "type": "message", "role": "user", "content": "Hello!" }
      ]
    }

  • Multipart content: messages with mixed content types

    {
      "input": [
        {
          "type": "message",
          "role": "user",
          "content": [
            { "type": "input_text", "text": "What's in this image?" },
            { "type": "input_image", "image_url": "https://example.com/image.png" },
            { "type": "input_file", "filename": "report.pdf", "file_data": "data:application/pdf;base64,JVBERi0xLjQK..." }
          ]
        }
      ]
    }

  • Function call outputs: tool result messages

    {
      "input": [
        { "type": "function_call_output", "call_id": "call_123", "output": "Paris, 15 degrees" }
      ]
    }

Streaming​

When streaming is enabled, the response uses Server-Sent Events with the Responses API event protocol:

event: response.created
data: {"type":"response.created","response":{...}}

event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_xxx","output_index":0,"content_index":0,"delta":"The capital"}

event: response.output_text.delta
data: {"type":"response.output_text.delta","item_id":"msg_xxx","output_index":0,"content_index":0,"delta":" of France is Paris."}

event: response.completed
data: {"type":"response.completed","response":{...}}

The full lifecycle events are emitted: response.created, response.in_progress, response.output_item.added, response.content_part.added, response.output_text.delta, response.output_text.done, response.content_part.done, response.output_item.done, response.completed.

Using Open Responses instead​

When use_open_response_for_responses is set to true, the /responses endpoint uses the Open Responses proxy implementation instead of the default one. The Open Responses proxy provides a richer implementation with native support for function calling, reasoning events, and more streaming events.

Regardless of this flag, the dedicated endpoints are always available:

  • /open-responses always uses the Open Responses proxy
  • /oai-responses always uses the default OpenAI Responses proxy

POST /messages​

Anthropic Messages API endpoint. This allows any Anthropic API client (including Claude Code) to use any LLM provider managed by Otoroshi.

curl https://api.domain.tld/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "gpt-4o",
    "max_tokens": 1024,
    "messages": [
      { "role": "user", "content": "Hello!" }
    ]
  }'

Response (Anthropic format):

{
  "id": "msg_xxxxx",
  "type": "message",
  "role": "assistant",
  "model": "gpt-4o",
  "content": [
    {
      "type": "text",
      "text": "Hello! How can I assist you today?"
    }
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 12,
    "output_tokens": 10
  }
}

The plugin handles all format translation automatically:

  • Anthropic-format tools (input_schema) are converted to OpenAI-format tools (parameters)
  • tool_use / tool_result messages are translated bidirectionally
  • Top-level system field is converted to a system message
  • thinking parameters are mapped to reasoning_effort
  • max_tokens is mapped to max_completion_tokens
  • Streaming uses the full Anthropic SSE protocol

Using Claude Code​

You can use Claude Code with this plugin by setting:

export ANTHROPIC_AUTH_TOKEN=your-otoroshi-api-key
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=https://api.domain.tld
claude --model gpt-4o

POST /embeddings​

OpenAI Embeddings API endpoint.

curl https://api.domain.tld/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "text-embedding-3-small",
    "input": "The quick brown fox jumps over the lazy dog"
  }'

POST /images/generations​

OpenAI Image Generation API endpoint.

curl https://api.domain.tld/v1/images/generations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "dall-e-3",
    "prompt": "A white cat sitting on a windowsill",
    "n": 1,
    "size": "1024x1024"
  }'

When decode_images is enabled, base64-encoded image results are decoded and returned as binary image data.

POST /images/edits​

OpenAI Image Edit API endpoint. Accepts multipart/form-data uploads.

POST /audio/speech​

OpenAI Text-to-Speech API endpoint.

curl https://api.domain.tld/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "tts-1",
    "input": "Hello, how are you?",
    "voice": "alloy"
  }' --output speech.mp3

POST /audio/transcriptions​

OpenAI Speech-to-Text API endpoint. Accepts multipart/form-data audio file uploads up to the configured max_size_upload limit.

POST /audio/translations​

OpenAI Audio Translation API endpoint. Translates audio into English text.

POST /moderations​

OpenAI Moderation API endpoint.

curl https://api.domain.tld/v1/moderations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "input": "Some text to check for moderation"
  }'

POST /ocr​

OCR endpoint that extracts text from images and PDF documents. Backed by the OCR model entities referenced in ocr_model_refs. Accepts either a JSON body (Mistral-style document) or a multipart/form-data file upload. See the OCR Models documentation for the full request and response formats.

curl https://api.domain.tld/v1/ocr \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -d '{
    "model": "alpha-digit-max",
    "document": {
      "type": "document_url",
      "document_url": "https://example.com/scan.pdf"
    }
  }'

Response:

{
  "model": "alpha-digit-max",
  "text": "The full extracted text...",
  "pages": [
    { "index": 0, "markdown": "The full extracted text..." }
  ],
  "usage_info": { "pages_processed": 1 }
}

POST /mcp​

Serves the MCP virtual server named by mcp_server_ref over the Streamable HTTP transport, exactly like the MCP exposition plugins — same protocol revisions, same tools, resources and prompts. The tools of your models and the tools of your MCP clients then live on one route, behind one API key, and the calls of both show up in the same analytics.

curl https://api.domain.tld/v1/mcp \
  -H "Authorization: Bearer $OTOROSHI_BEARER" \
  -H "Content-Type: application/json" \
  -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/list" }'

Set mcp_server_ref to expose it: with no reference, or when the referenced virtual server is disabled or gone, /mcp answers 404 like any unknown path of the API.

info

MCP clients expect an OAuth authorization server (RFC 9728) and will not send an API key on their own. Either configure the client with an Authorization header, or expose the same virtual server on a route of its own with enforce_oauth — see the MCP exposition plugins.

Securing the API​

Since this plugin exposes a powerful multi-endpoint AI API, you will want to secure it properly. Otoroshi provides several authentication mechanisms that can be combined on the same route. The key idea is that all security plugins run before the backend call, so you can stack multiple authentication methods and let the first valid one through.

Below are three common approaches, from simplest to most enterprise-ready. They can be used individually or combined on the same route.

API Keys​

The simplest approach. Otoroshi API keys support multiple extraction methods (header, query param, Basic auth, Bearer token, JWT) and come with built-in quotas and rate limiting.

Add the ApikeyCalls plugin to your route:

{
  "enabled": true,
  "plugin": "cp:otoroshi.next.plugins.ApikeyCalls",
  "config": {
    "extractors": {
      "basic": { "enabled": true },
      "custom_headers": { "enabled": true },
      "client_id": { "enabled": true },
      "jwt": { "enabled": true, "secret_signed": true, "keypair_signed": true },
      "oto_bearer": { "enabled": true }
    },
    "validate": true,
    "mandatory": true,
    "wipe_backend_request": true,
    "update_quotas": true
  }
}

Clients can then authenticate with a Bearer token:

curl https://api.domain.tld/v1/chat/completions \
  -H "Authorization: Bearer your-otoroshi-apikey" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}]}'

Or with Basic auth, client ID/secret headers, or JWT tokens signed with the API key secret.

You can further restrict access using mandatory tags to ensure only API keys with specific tags can access this route:

{
  "enabled": true,
  "plugin": "cp:otoroshi.next.plugins.NgApikeyMandatoryTags",
  "config": {
    "tags": ["endpoint_ai-api"]
  }
}

Biscuit tokens​

Biscuit tokens provide fine-grained, decentralized authorization with attenuation capabilities. This is ideal for distributing scoped tokens to different teams or applications.

Add the BiscuitUserExtractor plugin:

{
  "enabled": true,
  "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.biscuit.plugins.BiscuitUserExtractor",
  "config": {
    "keypair_ref": "biscuit-keypair_your-keypair-id",
    "enforce": true,
    "extractor_type": "header",
    "extractor_name": "Authorization",
    "validations": {
      "policies": [
        "allow if endpoint(\"ai-api\")"
      ]
    }
  }
}

Clients send their biscuit token in the Authorization header:

curl https://api.domain.tld/v1/chat/completions \
  -H "Authorization: biscuit_token_here" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}]}'

The biscuit policy allow if endpoint("ai-api") ensures the token was explicitly granted access to this endpoint. You can create attenuated tokens with additional restrictions (time limits, IP ranges, specific models, etc.).

OIDC / JWT (Keycloak, Auth0, etc.)​

For enterprise environments with an existing identity provider, you can validate JWT tokens issued by any OIDC-compliant provider (Keycloak, Auth0, Okta, Azure AD, etc.).

Add the OIDCJwtVerifier plugin:

{
  "enabled": true,
  "plugin": "cp:otoroshi.next.plugins.OIDCJwtVerifier",
  "config": {
    "ref": "auth_mod_your-oidc-verifier-id",
    "mandatory": true,
    "user": true
  }
}

The ref points to an Otoroshi JWT verifier entity configured with your OIDC provider's JWKS URL and issuer. Clients send their JWT token as a Bearer token:

# First, get a token from your OIDC provider
TOKEN=$(curl -s -X POST https://keycloak.your-domain.com/realms/your-realm/protocol/openid-connect/token \
  -d "grant_type=client_credentials" \
  -d "client_id=ai-api-client" \
  -d "client_secret=your-client-secret" | jq -r '.access_token')

# Then call the AI API
curl https://api.domain.tld/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}]}'

Combining multiple authentication methods​

The real power of Otoroshi is the ability to combine these methods on the same route. By setting mandatory: false on each auth plugin and adding the NgExpectedConsumer plugin, you can accept any of the configured authentication methods:

{
  "plugins": [
    {
      "enabled": true,
      "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.OpenAiCompatApi",
      "config": {
        "language_model_refs": ["provider_openai_1"],
        "embedding_model_refs": ["embedding_model_1"]
      }
    },
    {
      "enabled": true,
      "plugin": "cp:otoroshi.next.plugins.ApikeyCalls",
      "config": {
        "extractors": {
          "basic": { "enabled": true },
          "custom_headers": { "enabled": true },
          "jwt": { "enabled": true, "secret_signed": true, "keypair_signed": true },
          "oto_bearer": { "enabled": true }
        },
        "validate": true,
        "mandatory": false,
        "wipe_backend_request": true,
        "update_quotas": true
      }
    },
    {
      "enabled": true,
      "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.biscuit.plugins.BiscuitUserExtractor",
      "config": {
        "keypair_ref": "biscuit-keypair_your-keypair-id",
        "enforce": false,
        "extractor_type": "header",
        "extractor_name": "Authorization",
        "validations": {
          "policies": ["allow if endpoint(\"ai-api\")"]
        }
      }
    },
    {
      "enabled": true,
      "plugin": "cp:otoroshi.next.plugins.OIDCJwtVerifier",
      "config": {
        "ref": "auth_mod_your-oidc-verifier-id",
        "mandatory": false,
        "user": true
      }
    },
    {
      "enabled": true,
      "plugin": "cp:otoroshi.next.plugins.NgExpectedConsumer",
      "config": {}
    }
  ]
}

With this setup:

  • A developer can use a simple API key for quick prototyping
  • An automated pipeline can use a scoped biscuit token with fine-grained permissions
  • A web application can use a JWT token from your corporate Keycloak/Auth0
  • The NgExpectedConsumer plugin ensures that at least one authentication method succeeded

All three methods work on the same route, on the same API surface, without any change to the AI plugin configuration.

Comparison with individual plugins​

FeatureIndividual pluginsOpenAI Compatible API
Setup complexityOne plugin per endpointSingle plugin for everything
Path-based routingManual with includes filtersAutomatic based on URL path
Anthropic Messages supportSeparate AnthropicCompatProxy pluginBuilt-in via /messages
Responses API supportSeparate OpenAiResponsesProxy pluginBuilt-in via /responses
Open Responses supportSeparate OpenResponseCompatProxy pluginBuilt-in via /open-responses
Prompt contextsNot availableBuilt-in via /contexts
Model-specific configSame refs for all endpointsSeparate refs per capability (language, audio, image, embedding, moderation)
FlexibilityFine-grained control per endpointUnified but less granular

The individual plugins are still useful when you need fine-grained control per endpoint (e.g., different provider refs for different paths, or different plugin chains per endpoint). The unified plugin is ideal when you want a quick, coherent API surface with minimal configuration.