📄️ Overview
⚡ Cache: Fast and Efficient
📄️ Managing tokens usage
Our highly flexible quota management system ensures optimal resource allocation.
📄️ 💰 Cost tracking
Cost tracking for LLMs with a gateway means monitoring and managing the costs of using different LLMs through an API gateway.
📄️ Simple cache
simple cache works on prompts word per word
📄️ Semantic cache
semantic cache uses an embedding datastore to find prompt with the same semantic