AI Agent Node

The AI Agent node is the primary way to run an AI agent within an Otoroshi workflow. It acts as an autonomous agent that uses an LLM provider to reason and take actions.

Node kind: extensions.com.cloud-apim.llm-extension.ai_agent

Configuration

Parameter	Type	Required	Description
`name`	string	yes	Name of the agent
`provider`	string	yes	Id of the LLM provider to use
`description`	string	yes	Description of the agent (used for handoff descriptions)
`instructions`	array/string	yes	System instructions that define agent behavior
`input`	string/array/object	yes	The agent input - can be a text string, a messages array, or an expression language reference
`model`	string	no	Override the model used by the provider
`model_options`	object	no	Override model options (temperature, etc.)
`tools`	array	no	List of tool function ids the agent can use
`mcp_connectors`	array	no	List of MCP connector ids the agent can use
`inline_tools`	array	no	List of inline tool definitions (see below)
`memory`	string	no	Persistent memory provider id for conversation history
`guardrails`	array	no	List of guardrail configurations
`handoffs`	array	no	List of handoff configurations to other agents
`run_config`	object	no	Runtime configuration
`run_config.max_turns`	integer	no	Maximum number of agent turns (default: `10`)

Basic example

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "A helpful general-purpose assistant",
  "instructions": [
    "You are a helpful assistant that answers questions clearly and concisely."
  ],
  "input": "${input.question}",
  "result": "agent_response"
}

Using tools

Agents can use three types of tools:

Tool functions

Reference existing tool functions registered in the LLM extension by their id:

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "An assistant with tool access",
  "instructions": ["You help users by calling appropriate tools."],
  "input": "${input.question}",
  "tools": ["tool-function_xxxxx", "tool-function_yyyyy"],
  "result": "agent_response"
}

MCP connectors

Reference MCP connectors to give the agent access to MCP server tools:

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "An assistant with MCP access",
  "instructions": ["You help users by using available MCP tools."],
  "input": "${input.question}",
  "mcp_connectors": ["mcp-connector_xxxxx"],
  "result": "agent_response"
}

Inline tools

Define tools directly within the agent configuration. Each inline tool runs a workflow node when called:

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "An assistant with inline tools",
  "instructions": ["You help users by calling tools when needed."],
  "input": "${input.question}",
  "inline_tools": [
    {
      "name": "get_current_time",
      "description": "Returns the current date and time",
      "parameters": {
        "type": "object",
        "properties": {
          "timezone": {
            "type": "string",
            "description": "The timezone (e.g. Europe/Paris)"
          }
        }
      },
      "required": ["timezone"],
      "node": {
        "kind": "call",
        "function": "core.system_call",
        "args": {
          "command": ["date"]
        }
      }
    }
  ],
  "result": "agent_response"
}

When the LLM decides to call an inline tool, the arguments are stored in the workflow memory as tool_input, and the associated workflow node is executed. The node result is returned to the LLM as the tool result.

You can set response_json_parse: true (or input_json_parse: true) on an inline tool to parse the tool arguments as JSON before storing them in tool_input.

Built-in tools

Agents can be equipped with a library of built-in tools that require no external configuration. These tools are enabled per-agent via the built_in_tools configuration and cover the core capabilities an autonomous agent typically needs: filesystem access, shell commands, HTTP calls, task tracking, planning, working memory, and agent delegation.

Configuration

Parameter	Type	Description
`all`	boolean	Enable all tool categories at once (default: `false`)
`workspace`	boolean	Enable file tools: `list_files`, `read_file`, `write_file`, `append_file`, `search_in_files`
`shell`	boolean	Enable `run_command` tool
`http`	boolean	Enable `http_call` tool
`content_to_markdown`	boolean	Enable `content_to_markdown` tool (document conversion via Kreuzberg)
`tasks`	boolean	Enable `task_create`, `task_update`, `task_get`, `task_list`
`plan`	boolean	Enable `plan_set`, `plan_get`
`memory`	boolean	Enable `memory_set`, `memory_get`, `memory_list`
`persistent_kv`	boolean	Enable persistent KV tools: `persistent_kv_set`, `persistent_kv_get`, `persistent_kv_delete`, `persistent_kv_list`
`persistent_kv_uri`	string	Connection URI for the persistent KV backend (`redis://...` or `postgresql://...`)
`persistent_kv_namespace`	string	Namespace for key isolation (supports Otoroshi EL expressions, default: `agent-kv:{agent_name}:default`)
`agent`	boolean	Enable `delegate` tool (delegates to handoff agents)
`control`	boolean	Enable `final_answer` tool
`include`	array	Enable specific tools by name (overrides category flags)
`exclude`	array	Disable specific tools by name (overrides everything)
`allowed_paths`	array	Whitelist of filesystem paths the agent can access (required for workspace/shell tools)
`command_timeout`	integer	Shell command timeout in milliseconds (default: `30000`)
`sub_agents`	array	Sub-agent profiles for the `spawn_agent` tool (see below)

Example: autonomous coding agent

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "coder",
  "provider": "provider_xxxxx",
  "description": "An autonomous coding agent",
  "instructions": [
    "You are an autonomous coding agent.",
    "Use the available tools to explore the codebase, understand it, and make changes.",
    "Always read files before modifying them."
  ],
  "input": "${input.question}",
  "built_in_tools": {
    "workspace": true,
    "shell": true,
    "tasks": true,
    "plan": true,
    "memory": true,
    "allowed_paths": ["/workspace/my-project"],
    "command_timeout": 30000
  },
  "run_config": {
    "max_turns": 20
  },
  "result": "agent_response"
}

Example: read-only analyzer with HTTP access

{
  "built_in_tools": {
    "workspace": true,
    "http": true,
    "tasks": true,
    "memory": true,
    "allowed_paths": ["/workspace/my-project"],
    "exclude": ["write_file", "append_file"]
  }
}

Example: fine-grained tool selection

{
  "built_in_tools": {
    "include": ["read_file", "search_in_files", "http_call", "final_answer"],
    "allowed_paths": ["/workspace/my-project"]
  }
}

Built-in tool reference

Category	Tool	Description
Workspace	`list_files`	List files and directories at a given path
Workspace	`read_file`	Read a text file content
Workspace	`write_file`	Write content to a file (creates parent dirs)
Workspace	`append_file`	Append content to a file
Workspace	`search_in_files`	Search for a text pattern across files
Shell	`run_command`	Run a shell command in the workspace
HTTP	`http_call`	Make an HTTP request (GET, POST, PUT, DELETE, etc.)
Content	`content_to_markdown`	Convert a document (PDF, DOCX, HTML, etc.) to markdown via Kreuzberg
Persistent KV	`persistent_kv_set`	Store a key-value pair in persistent memory (survives across sessions)
Persistent KV	`persistent_kv_get`	Read a value from persistent memory
Persistent KV	`persistent_kv_delete`	Delete a key from persistent memory
Persistent KV	`persistent_kv_list`	List all entries in persistent memory
Tasks	`task_create`	Create a task to track work
Tasks	`task_update`	Update a task's status or fields
Tasks	`task_get`	Get a task by id
Tasks	`task_list`	List all tasks
Plan	`plan_set`	Set or replace the current working plan
Plan	`plan_get`	Get the current plan
Memory	`memory_set`	Store a key-value pair in working memory
Memory	`memory_get`	Read a value from working memory
Memory	`memory_list`	List all working memory entries
Agent	`delegate`	Delegate a subtask to a handoff agent
Agent	`spawn_agent`	Spawn a sub-agent from a pre-configured profile
Control	`final_answer`	Signal that the objective is complete

Workspace sandboxing

All file and shell operations are sandboxed to the paths listed in allowed_paths. Any attempt to access a path outside these directories is denied. Paths can be absolute or relative to the first allowed path.

Working memory (scratchpad)

When task, plan, or memory tools are enabled, the agent gets a scratchpad — a structured working memory injected into its system prompt. The scratchpad tracks:

Tasks: a list of sub-tasks with status (todo, in_progress, done, blocked)
Plan: a goal with ordered steps
Notes: key-value store for observations and intermediate results

The LLM uses these tools to organize multi-step work and track its own progress.

Content to markdown (Kreuzberg)

The content_to_markdown tool uses Kreuzberg to convert documents (PDF, DOCX, HTML, images, etc.) into markdown text. The agent can provide either a URL or base64-encoded content.

{
  "built_in_tools": {
    "content_to_markdown": true
  }
}

The tool accepts url (fetches and converts the document) or content + content_type (base64-encoded bytes with MIME type).

Persistent KV memory

The persistent KV tools give agents a key-value store that survives across sessions. Unlike the working memory (scratchpad), which is ephemeral to the current agent run, persistent KV data is stored externally in Redis or PostgreSQL.

The backend is auto-detected from the URI scheme:

redis:// or rediss:// — uses a Redis HASH per namespace (HSET/HGET/HDEL/HGETALL)
postgresql:// — uses a PostgreSQL table otoroshi_agent_kv (created automatically)

{
  "built_in_tools": {
    "persistent_kv": true,
    "persistent_kv_uri": "redis://localhost:6379",
    "persistent_kv_namespace": "agent-kv:${consumer.id}:my-agent"
  }
}

The persistent_kv_namespace isolates data between agents and sessions. It supports Otoroshi expression language variables:

Expression	Description
`${consumer.id}`	The consumer id
`${apikey.id}`	The API key id
`${user.email}`	The authenticated user email
`${token.sub}`	The JWT subject
`${req.ip}`	The client IP address

Example with PostgreSQL and per-apikey isolation:

{
  "built_in_tools": {
    "persistent_kv": true,
    "persistent_kv_uri": "postgresql://user:pass@localhost:5432/mydb",
    "persistent_kv_namespace": "agent-kv:${apikey.id}:assistant"
  }
}

Sub-agent spawning

The spawn_agent tool lets the LLM create sub-agents on the fly. Unlike delegate (which routes to a pre-configured handoff agent), spawn_agent lets the LLM write the instructions at call time. Sub-agent profiles define the capabilities (tools, provider, model), and the LLM provides the instructions and objective.

{
  "built_in_tools": {
    "all": true,
    "allowed_paths": ["/workspace"],
    "sub_agents": [
      {
        "name": "researcher",
        "description": "Agent for code analysis and research",
        "built_in_tools": {
          "workspace": true,
          "memory": true,
          "allowed_paths": ["/workspace"]
        }
      },
      {
        "name": "implementer",
        "description": "Agent for writing and modifying code",
        "built_in_tools": {
          "workspace": true,
          "shell": true,
          "allowed_paths": ["/workspace"]
        }
      }
    ]
  }
}

The LLM calls the tool with objective, instructions (the system prompt it writes), and optionally agent_profile (which profile to use). Sub-agents inherit the parent's provider/model if the profile doesn't specify one. Sub-agents cannot spawn further sub-agents (no recursive spawning).

Agent handoffs

Handoffs allow an agent to transfer the conversation to another specialized agent. The triage agent sees each possible handoff as a tool it can call.

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "triage_agent",
  "provider": "provider_xxxxx",
  "description": "A triage agent that routes questions",
  "instructions": [
    "You determine which agent to use based on the user's question"
  ],
  "input": "${input.question}",
  "handoffs": [
    {
      "enabled": true,
      "agent": {
        "name": "math_tutor",
        "provider": "provider_xxxxx",
        "description": "Specialist agent for math questions",
        "instructions": [
          "You provide help with math problems. Explain your reasoning at each step."
        ],
        "mcp_connectors": []
      }
    },
    {
      "enabled": true,
      "agent": {
        "name": "history_tutor",
        "provider": "provider_xxxxx",
        "description": "Specialist agent for historical questions",
        "instructions": [
          "You provide assistance with historical queries. Explain events and context clearly."
        ],
        "mcp_connectors": []
      }
    }
  ],
  "result": "agent_response"
}

Handoff configuration

Parameter	Type	Required	Description
`enabled`	boolean	no	Whether this handoff is active (default: `true`)
`agent`	object	yes	The target agent configuration (same structure as the AI Agent node)
`tool_name_override`	string	no	Override the tool name (default: `transfer_to_<agent_name>`)
`tool_description_override`	string	no	Override the tool description

When a handoff occurs, the LLM calls a function named transfer_to_<agent_name> (or the overridden name). The agent runner then executes the target agent with the same input, inheriting the provider and model configuration if not overridden.

Persistent memory

Connect an agent to a persistent memory provider to maintain conversation history across workflow executions:

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "An assistant with memory",
  "instructions": ["You are a helpful assistant."],
  "input": "${input.question}",
  "memory": "memory_xxxxx",
  "result": "agent_response"
}

Guardrails

Apply guardrails to validate agent inputs and outputs:

{
  "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
  "name": "assistant",
  "provider": "provider_xxxxx",
  "description": "A safe assistant",
  "instructions": ["You are a helpful assistant."],
  "input": "${input.question}",
  "guardrails": [
    {
      "id": "regex",
      "before": true,
      "after": false,
      "config": {
        "deny": [".*password.*", ".*credit card.*"]
      }
    },
    {
      "id": "prompt_injection",
      "before": true,
      "after": false,
      "config": {
        "provider": "provider_xxxxx"
      }
    }
  ],
  "result": "agent_response"
}

Available guardrail kinds

regex, webhook, llm, secrets_leakage, auto_secrets_leakage, gibberish, pif, moderation, moderation_model, toxic_language, racial_bias, gender_bias, personal_health_information, prompt_injection, faithfulness, sentences, words, characters, contains, semantic_contains, quickjs, wasm

Complete example: multi-agent system with tools

{
  "kind": "workflow",
  "steps": [
    {
      "kind": "extensions.com.cloud-apim.llm-extension.ai_agent",
      "name": "triage",
      "provider": "provider_xxxxx",
      "description": "A triage agent",
      "instructions": [
        "Route the user question to the appropriate specialist agent."
      ],
      "input": "${input.question}",
      "handoffs": [
        {
          "enabled": true,
          "agent": {
            "name": "researcher",
            "provider": "provider_xxxxx",
            "description": "Research agent with web search capabilities",
            "instructions": [
              "You help users find information using web search tools."
            ],
            "mcp_connectors": ["mcp-connector_brave-search"]
          }
        },
        {
          "enabled": true,
          "agent": {
            "name": "coder",
            "provider": "provider_xxxxx",
            "description": "Coding assistant",
            "instructions": [
              "You help users with coding problems. Write clean, documented code."
            ],
            "tools": ["tool-function_code-executor"],
            "mcp_connectors": []
          }
        }
      ],
      "guardrails": [
        {
          "id": "prompt_injection",
          "before": true,
          "after": false,
          "config": { "provider": "provider_xxxxx" }
        }
      ],
      "result": "agent_response"
    }
  ],
  "returned": "${agent_response}"
}

Configuration​

Basic example​

Using tools​

Tool functions​

MCP connectors​

Inline tools​

Built-in tools​

Configuration​

Example: autonomous coding agent​

Example: read-only analyzer with HTTP access​

Example: fine-grained tool selection​

Built-in tool reference​

Workspace sandboxing​

Working memory (scratchpad)​

Content to markdown (Kreuzberg)​

Persistent KV memory​

Sub-agent spawning​

Agent handoffs​

Handoff configuration​

Persistent memory​

Guardrails​

Available guardrail kinds​

Complete example: multi-agent system with tools​

Workflow editor with an autonomous agent and its sub-agents​

Configuration

Basic example

Using tools

Tool functions

MCP connectors

Inline tools

Built-in tools

Configuration

Example: autonomous coding agent

Example: read-only analyzer with HTTP access

Example: fine-grained tool selection

Built-in tool reference

Workspace sandboxing

Working memory (scratchpad)

Content to markdown (Kreuzberg)

Persistent KV memory

Sub-agent spawning

Agent handoffs

Handoff configuration

Persistent memory

Guardrails

Available guardrail kinds

Complete example: multi-agent system with tools

Workflow editor with an autonomous agent and its sub-agents