Prompt templating

Prompt templates let you define standardized message sets with dynamic expression placeholders. When a template is active, it replaces the consumer's messages entirely, giving you full control over what gets sent to the LLM.

How it works

1. The consumer sends a request to the route
2. The prompt template plugin intercepts the request and resolves the template
3. Expression placeholders (@{path}) are replaced with values from the request context
4. The rendered messages replace the consumer's original messages
5. The request is forwarded to the LLM provider with the templated messages

Template syntax

A template is a JSON array of message objects with @{path} expression placeholders:

[
  {
    "role": "system",
    "content": "You are a helpful assistant for the @{request.route.name} service."
  },
  {
    "role": "user",
    "content": "@{body.text}"
  }
]

Expression format

Expressions use the @{path} syntax where path is a JSON path resolved against the template context object.

The context object contains:

Path prefix	Description
body.*	The incoming request body fields
request.*	The Otoroshi request context (route, headers, query params, etc.)

Expression examples

Expression	Description
@{body.text}	The text field from the request body
@{body.messages[0].content}	First message content from the request body
@{request.route.name}	Route name
@{request.route.id}	Route ID

Entity structure

{
  "id": "template_xxx",
  "name": "Translation template",
  "description": "Translates the body text field to English",
  "template": "[{\"role\": \"system\", \"content\": \"Translate the following text to English.\"}, {\"role\": \"user\", \"content\": \"@{body.text}\"}]"
}

Plugin configuration

Add the Cloud APIM - LLM Proxy - prompt template plugin to your route, before the LLM proxy plugin:

{
  "enabled": true,
  "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.AiPromptTemplate",
  "config": {
    "ref": "template-entity-id"
  }
}

Parameter	Type	Description
ref	string	Reference to a PromptTemplate entity ID

Important: When using templates, disable allow_config_override on the provider to prevent the consumer from bypassing the template by sending their own messages:

Example

Template

[
  {"role": "system", "content": "You are a travel assistant. Answer questions about destinations."},
  {"role": "user", "content": "Tell me about @{body.destination} for a @{body.duration} trip."}
]

Consumer request

curl --request POST \
  --url http://myroute.oto.tools:8080/ \
  --header 'content-type: application/json' \
  --data '{
  "destination": "Japan",
  "duration": "2 weeks"
}'

Result sent to LLM

[
  {"role": "system", "content": "You are a travel assistant. Answer questions about destinations."},
  {"role": "user", "content": "Tell me about Japan for a 2 weeks trip."}
]

Testing templates

The admin UI includes a built-in template tester. Provide a JSON context object and test the template rendering + LLM call directly from the Otoroshi back-office.