Provider agnostic

The Otoroshi LLM Extension is able to connect with popular LLM providers such as OpenAI, Mistral, Anthropic, Azure, Ollama, etc. while providing a common API for consumers.

Prompt engineering

The Otoroshi LLM Extension offer various way to customize your prompts and implements LLM compliance and governance.

Monitor and secure usage of LLMs

Secure access by providing fine grained apikeys with specific access rights. Monitor and limit usages of specific providers to avoid unexpected costs.

Unified interface

Simplify interactions and minimize integration hassles. Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API

Prompt fences

Validate your prompts and prompts responses to avoid sensitive or personal informations leakage, irrelevant or unhelpful responses, gibberish content, etc

Cloud APIM integration

The Otoroshi LLM Extension is available on Cloud APIM managed instance and Cloud APIM Serverless.