OpenAI decisions API
Your decision models also answer the decisions API of OpenAI. An application written with the OpenAI SDKs calls client.decisions.create(...) against the gateway, and the decision is made by the model you chose for it: Jev, Clef, a model you host, or one of your LLMs. Same entities, same API keys, same budgets and cost tracking as the System One endpoint.
from openai import OpenAI
client = OpenAI(
base_url="https://api.domain.tld/v1",
api_key="your-otoroshi-api-key",
)
result = client.decisions.create(
model="jev-latest",
input="The package arrived with a broken screen. I want my money back.",
questions=[
{"type": "predicate", "name": "damaged", "instructions": "Does the customer report a damaged item?"},
{
"type": "choice",
"name": "action",
"instructions": "What should we do?",
"choices": [
{"value": "refund", "description": "Refund the order"},
{"value": "replace", "description": "Send a new one"},
{"value": "escalate", "description": "Ask a human"},
],
},
],
)
damaged, action = result.answers
print(damaged.probability)
print(action.choice)
Endpoints
| Plugin | Path |
|---|---|
| LLM OpenAI Compatible API | /v1/decisions, next to /v1/systemone and the other endpoints, with the decision models of decision_model_refs |
| OpenAI decisions backend | Any path of its route, with the decision models of its refs |
{
"enabled": true,
"plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.OpenAICompatDecisions",
"config": {
"refs": ["decision-model-entity-id"]
}
}
The model of a request picks the decision model and the model it is asked for, with the same rules as the System One endpoint. Without a model, the first decision model answers with its own.
Request
curl https://api.domain.tld/v1/decisions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OTOROSHI_BEARER" \
-d '{
"model": "jev-latest",
"input": "The package arrived with a broken screen.",
"questions": [
{ "type": "predicate", "name": "damaged", "instructions": "Does the customer report a damaged item?" },
{
"type": "choice",
"name": "refund",
"instructions": "Should we refund the order?",
"choices": [
{ "value": true, "description": "Refund it" },
{ "value": false, "description": "Do not refund it" }
]
},
{
"type": "score",
"name": "anger",
"instructions": "How angry is the customer?",
"levels": [
{ "label": "calm" },
{ "label": "upset", "description": "Complains but stays polite" },
{ "label": "furious" }
]
}
]
}'
| Parameter | Type | Description |
|---|---|---|
input | string or array | What the questions are about: a text, or a whole conversation |
questions | array | 1 to 400 questions. Each has a type, instructions (a text, or input_text, input_image and input_json parts) and an optional name |
model | string | The model to ask. Optional: the model of the decision model entity is used when omitted |
safety_identifier | string | The end user of your application, given to OpenAI for its abuse monitoring |
prompt_cache_key, prompt_cache_options | string, object | The prompt cache settings of OpenAI, with the prompt_cache_breakpoint of the parts |
| Type | Asks | With |
|---|---|---|
predicate | How likely a statement about the input is true | Nothing more |
choice | One value among several | choices: 2 to 255, each with a value, a string or a boolean, and an optional description |
score | A position on an ordered scale | levels: each label and an optional description, from the lowest to the highest |
Every question is asked in one call, against the same input. The gateway checks a request with the rules of OpenAI before any provider is called, and answers what it cannot read with a 400 naming the parameter at fault.
Conversations and tool calls
The input can be a whole conversation, written the way the Responses API writes it: messages of a user (input_text, input_image and input_json parts), of a developer or of the system, answers of the assistant (output_text and refusal parts), function calls and custom tool calls with their outputs. Ask whether an agent called the right tool, whether its answer holds to what the tool returned, whether the customer is satisfied, in one call:
{
"model": "gpt-6-luna",
"input": [
{ "role": "developer", "content": "You answer the customers of a shop." },
{ "role": "user", "content": "Where is my order 42?" },
{ "type": "function_call", "call_id": "c1", "name": "find_order", "arguments": "{\"id\":42}" },
{ "type": "function_call_output", "call_id": "c1", "output": "delivered yesterday" },
{ "role": "assistant", "content": "Your order is on its way." }
],
"questions": [
{ "type": "predicate", "name": "grounded", "instructions": "Does the last answer agree with what the tool returned?" }
]
}
Each output answers a call of the conversation, by its call_id. OpenAI receives the conversation as it was written. The other decision models read it as a transcript that names the author of each item:
[developer]
You answer the customers of a shop.
[user]
Where is my order 42?
[function call find_order (c1)]
{"id":42}
[function call output (c1)]
delivered yesterday
[assistant]
Your order is on its way.
An input_json part holds an object or an array of objects, read as the compact JSON it is, the way OpenAI reads it. Give it a record, an order, the result of a query, as they are.
Response
{
"model": "jev-latest",
"answers": [
{ "type": "predicate", "name": "damaged", "probability": 0.95 },
{
"type": "choice",
"name": "refund",
"choice": true,
"confidence": 0.62,
"probabilities": [
{ "value": true, "probability": 0.81 },
{ "value": false, "probability": 0.19 }
]
},
{
"type": "score",
"name": "anger",
"score": 1.12,
"confidence": 0.4,
"probabilities": [
{ "label": "calm", "value": 0, "probability": 0.14 },
{ "label": "upset", "value": 1, "probability": 0.6 },
{ "label": "furious", "value": 2, "probability": 0.26 }
]
}
],
"usage": {
"input_tokens": 96,
"input_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 0 },
"output_tokens": 0,
"output_tokens_details": { "reasoning_tokens": 0 },
"total_tokens": 96
}
}
The answers come in the order of the questions, each with the name it was given (null without one). A choice answers with the value as it was asked: a boolean choice is a boolean again. A question the model does not answer is a refusal, and the other questions are answered all the same:
{ "type": "refusal", "name": "anger" }
Add ?embed_costs=true to a request to get what it cost in a costs field.
One API, every decision model
The questions are the same as the ones of the System One API, in other words: a predicate is a noul, the choices of a choice and the levels of a score are its criteria, the input is the state. So every provider answers an OpenAI request, OpenAI itself receiving it exactly as it was written, the fallback works across all of them, and the guardrails and workflows built on your decision models keep working whatever API your applications call.
Pictures
input_image parts carry pictures, in the conversation as in the instructions of a question, as data URLs (data:image/png;base64,...) or as the https:// address they are at. They are shown to the decision models that look at them, like the ones of OpenAI, Jev of TypeSafe, Clef on Cloudflare Workers AI or a self-hosted server: each one receives them the way its API takes them, a picture of the instructions being named where it was ([image 2]) for the models shown every picture before the input.
OpenAI fetches a picture from its address, and a self-hosted server receives the address as it is. Jev and Clef take the picture itself: give it to them as a data URL. A model that cannot see a picture refuses the request with a 400: a decision is never made on a part of what it is about.
Errors
Errors come in the format of the OpenAI API, so the SDKs raise the exception they would raise against OpenAI:
{
"error": {
"message": "Invalid value: 'rank'. Supported values are: 'predicate', 'choice', 'score'.",
"type": "invalid_request_error",
"param": "questions[0].type",
"code": null
}
}
| Status | When |
|---|---|
400 | The request cannot be read, or it holds pictures for a model that reads text only, or the address of a picture for a model that takes the picture itself |
401, 422, 429, 529... | What the provider answered, said in the words of the OpenAI API. A Retry-After header is passed along |
402 | A budget is exhausted |
403 | The model is not one the consumer may use, or it has no known price while the decision model requires one |
504, 502 | The provider did not answer in time, or could not be reached, and no fallback took over |