Private AI inference and confidential GPU computing infrastructure
Open-source AI gateway you run yourself
Stage
Not given
Not given
Founded
Not given
Not given
Based in
US
Not given
Pricing model
Paid
Open source
Pricing
Confidential Inference: DeepSeek V4.1 Flash $0.30 input and $1.20 output per 1M tokens; Kimi K3 $3.00 input and $15.00 output per 1M tokens; GLM 5.3 $1.40 input and $4.40 output per 1M tokens; Qwen3.8 27B $0.30 input and $1.80 output per 1M tokens. Confidential VMs: B300 at $7.50 per GPU-hour, B200
Not given
Key features
OpenAI-compatible endpoint for private LLM inference
Hardware-encrypted VMs on Intel TDX, AMD SEV-SNP and NVIDIA GPUs
Client attestation verification before sending requests
Support for multiple open-weight models including DeepSeek, Kimi, GLM, Qwen
Confidential Kubernetes for data center scale deployments
Confidential training and fine-tuning on licensed stack
Root access over SSH to confidential VMs
Encrypted GPU communications with NVLink encryption on B200 and B300
Confidential agents runtime support
Prompt caching with cached input billing
Single sign-on with OpenID Connect and two-factor authentication
Format translation through KPP canonical format, allowing any SDK to call any model
Spend and token budgets with hard limits that are never skipped
Complete request logging with status, tokens, cost, timing and content when permitted
Encrypted provider key storage with AES-256-GCM
Web console for managing organizations, projects, members, roles and request logs
Stateless Rust gateway for horizontal scaling
Support for 14 built-in AI providers
Docker Compose deployment with Caddy, PostgreSQL, Valkey and ClickHouse
Error reporting that names unsupported fields instead of silently dropping them
What makes it different
Prompts and model weights verified by clients with cryptographic attestation before any data is sent
Operator-blind inference: data remains unreadable to the company and hosting provider
Flexible deployment: choice of hosted cloud, rented VMs, or licensed stack on own infrastructure
Open-weight models only, no closed-source frontier models
Full cryptographic verification stack including open-source tools
OpenID Connect single sign-on and two-factor authentication included in the open-source code, not an enterprise edition
Hard budgets with worst-case reserves for every request to prevent burst overruns
Single MIT-licensed repository for gateway, console, KPP and UI library with no separate enterprise version
Rust gateway architecture with no interpreter on the hot path for performance
Confidential AI
Kaeso