Skip to main content

HTTP response generator

The LLM Response generator plugin acts as a complete backend — it does not forward requests to any upstream service. Instead, it sends a pre-configured prompt to an LLM and uses the LLM's response as the HTTP response body.

How it works​

  1. A request arrives at the route
  2. The plugin sends the configured prompt to the LLM provider (the request body is not used)
  3. The LLM generates a response
  4. The LLM response is returned directly to the client as the HTTP response

This is useful for creating AI-powered endpoints that generate content entirely from a prompt, without any backend service.

Plugin configuration​

{
  "enabled": true,
  "plugin": "cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.AiResponseGenerator",
  "config": {
    "ref": "provider-entity-id",
    "prompt": "Generate a random motivational quote in JSON format: {\"quote\": \"...\", \"author\": \"...\"}",
    "prompt_ref": null,
    "context_ref": null,
    "is_response": false,
    "status": 200,
    "headers": {}
  }
}

Parameters​

ParameterTypeDefaultDescription
refstring""LLM Provider entity ID
promptstring""The prompt to send to the LLM
prompt_refstringnullReference to a stored prompt entity
context_refstringnullReference to a stored context entity for pre/post messages
is_responsebooleanfalseWhen true, the LLM response is parsed as a full HTTP response (see below)
statusnumber200HTTP status code for the response (used when is_response is false)
headersobject{}HTTP response headers (used when is_response is false)

Full HTTP response mode​

When is_response is set to true, the LLM is expected to return a complete HTTP response as a JSON object:

{
  "status": 200,
  "headers": {
    "Content-Type": "application/json"
  },
  "body": "{\"quote\": \"The only way to do great work is to love what you do.\", \"author\": \"Steve Jobs\"}"
}

The plugin will use these values to construct the actual HTTP response.

Example: AI-powered FAQ endpoint​

{
  "ref": "provider_openai",
  "prompt": "You are a helpful FAQ assistant for our product. Answer common questions about pricing, features, and support. Return a JSON object with a 'question' and 'answer' field.",
  "status": 200,
  "headers": {
    "Content-Type": "application/json"
  }
}

Example: dynamic content generation​

{
  "ref": "provider_openai",
  "prompt": "Generate an HTML page with a random poem about technology. Include basic CSS styling.",
  "is_response": true
}

The LLM would return:

{
  "status": 200,
  "headers": {
    "Content-Type": "text/html"
  },
  "body": "<html><head><style>body{font-family:serif;max-width:600px;margin:40px auto}</style></head><body><h1>Digital Dreams</h1><p>...</p></body></html>"
}