Pay-as-you-go pricing for serverless inference charged per token and for on-demand deployments charged per GPU usage time. New users receive free credits. Text models range from $0.05 to $1.5 per 1M tokens (input and output), including Llama-3.2-1B-Instruct at $0.06/1M tokens, Meta-Llama-3.1-8B-Inst
Not given
Key features
Text generation with multiple LLM models including Llama, Qwen, and DeepSeek
Embedding and reranking models for RAG applications
Text-to-image and text-to-video generation
Voice generation with ASR/TTS models
Single line of code integration
Agent application with ultra-low latency
Chat2DB application with real-time responsiveness
Inference acceleration capability
Competitive pricing with reduced costs
OpenAI-compatible API interface
Support for 40+ leading models including GPT, Claude, Gemini, DeepSeek, and Grok
Real-time intelligent routing based on availability, latency, cost and regional policy
Automatic failover with sub-second switchover to standby nodes
Continuous session management across model and provider switches
Real-time usage metering by model, project, member and API key
Team member and project quota management with role-based access control
Unified billing and comprehensive audit logging
Budget limits and quota controls across teams and projects
Multi-node high-availability deployment with health checks
What makes it different
Out-of-the-box inference acceleration capability
Fast model updates and comprehensive model selection
Single line of code integration with no deployment complexity
Ultra-low latency for responsive applications
Free credits for new users to get started
Single unified API key and endpoint replaces need to maintain separate credentials and integrations for each model provider
Intelligent routing automatically selects optimal model and provider based on live performance metrics
Automatic failover minimizes impact of provider outages by switching to available alternatives in milliseconds
Maintains conversation continuity when switching between models and providers
Minutes to integrate with existing code by only changing base URL and API key