LLM Gateway | Otoroshi LLM Extension

📄️ Philosophy

Today, generative AI is simply unavoidable. It’s transforming industries, unlocking new possibilities, and accelerating innovation

📄️ Supported LLM Providers

Here is a list of all providers available in the Otoroshi LLM Extension :

📄️ ⚙️ Setup a new LLM Provider

Open your Otoroshi admin web UI on otoroshi.oto.tools:8080

📄️ 👀 Expose your LLM Provider

now that your provider is fully setup, you can expose it to your organization. The idea here is to do it through an Otoroshi route

📄️ 🔐 Secure your LLM provider endpoint

Video Turorial

📄️ Managing secrets

You can use a secret vault to save your LLM provider tokens. The main advantage here is to avoid spreading those token and giving otoroshi apikeys to your own people instead.

📄️ Resilience

Resilience ensures that LLM interactions remain highly available and fault-tolerant, even when providers experience outages or failures.

📄️ Observability

Every interaction with a Large Language Model (LLM) generates crucial data that can be monitored, analyzed, and optimized.