TensorZero
tensorzero.com
- Free plan
- Linux
- Web
- API
- Self-hosted
What TensorZero is
TensorZero is an open-source stack for building and operating LLM applications. It combines a gateway with observability, optimization, evaluation, and experimentation. The gateway offers a unified API for documented model providers including Anthropic, AWS Bedrock, Azure, Google AI Studio, OpenAI, OpenRouter, vLLM, and xAI, as well as OpenAI-compatible APIs. It supports routing, retries, fallbacks, prompt templates, schemas, multi-step workflows, and A/B tests. Inference and feedback data go to a database you configure and can query with SQL; feedback can be tied to individual inferences or episodes. Deployment options include Docker, Docker Compose, Kubernetes with Helm, or building from source. API key authentication can protect gateway endpoints other than status and health checks. The gateway also collects pseudonymous aggregate usage analytics, which it says exclude application data and inference inputs and outputs; collection can be disabled. TensorZero is free and self-hosted under the Apache 2.0 License, but its homepage says the project is no longer maintained, and GitHub marks the repository archived and read-only.
Who it suits
TensorZero may suit teams building LLM applications that want configurable routing and SQL access to inference and feedback data. It is less suitable for teams that need a maintained project, since its homepage says development has stopped.
Good to know, and what to watch
- Unified API for documented providers and compatible endpoints
- Supports fallbacks, retries, and traffic routing
- Stores inference and feedback data in your database
- Deployable with Docker, Helm, or from source
- No longer maintained; GitHub repository is archived
- Requires a self-hosted deployment
- Usage analytics are collected by default, though configurable
Plans
Every plan and priceTensorZero LLMOps platform
Free plan
The verdict
TensorZero brings LLM routing, observability, and experimentation into a self-hosted stack with data stored in your database. Its archived, unmaintained status is a significant consideration for teams evaluating it.
What the maker publishes
- Clients
- TensorZero supports the OpenAI SDK, OpenTelemetry, and clients for major programming languages. source
- Company
- The FAQ describes the team as based in New York City and names Viraj Mehta and Gabriel Bianconi as its founders. source
- Current status
- The homepage says TensorZero remains available on GitHub but is no longer maintained. source
- Data control
- TensorZero documentation says it stores its data in the user's own database, which can be queried directly with SQL. source
- Deployment
- The gateway and UI can be deployed with Docker, Docker Compose, or Kubernetes and Helm, or built from source. source
- Deployment
- The gateway can be deployed with Docker, Docker Compose, Kubernetes using a reference Helm chart, or built from source with Cargo. source
- Experiments and workflows
- The gateway supports multi-step workflows and traffic routing for A/B tests, with feedback assignable to individual inferences or episodes. source
- Features
- The product includes routing, retries, fallbacks, prompt templates, schemas, and A/B testing. source
- Gateway
- The gateway provides a unified API for major LLM providers, with fallbacks and reported sub-millisecond P99 latency overhead under extreme load. source
- Gateway
- The gateway provides a unified API for accessing supported LLM providers. source
- Integrations
- Documented providers include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, and OpenAI-compatible endpoints. source
- License
- The FAQ describes TensorZero as open source under the Apache 2.0 License. source
- Maintenance status
- The official site says TensorZero remains available on GitHub but is no longer maintained. source
- Model integrations
- Documented providers include Anthropic, AWS Bedrock, Azure, Google AI Studio, OpenAI, OpenRouter, vLLM, and xAI; OpenAI-compatible APIs are also supported. source
- Notable limitation
- The FAQ says TensorZero had no revenue model at the time described and planned a managed service for the future; the current homepage says the project is no longer maintained. source
- Observability
- The gateway stores inference and feedback data in the configured database for observability, experimentation, and optimization. source
- Observability
- TensorZero can store inference and feedback data for observability, optimization, evaluations, and experimentation in Postgres or ClickHouse. source
- Privacy
- The gateway documentation says its pseudonymous usage analytics do not include application data or inference inputs and outputs, and can be disabled. source
- Product
- TensorZero is described in its documentation as an open-source stack combining an LLM gateway, observability, optimization, evaluation, and experimentation. source
- Product
- TensorZero is an open-source platform that combines an LLM gateway, observability, evaluation, optimization, and experimentation. source
- Project status
- GitHub marks the TensorZero repository as archived and read-only, archived on June 12, 2026. source
- Security
- The gateway supports API key authentication, which can be enabled to protect its endpoints other than status and health checks. source
- Security
- TensorZero's privacy policy says it uses physical, technical, organizational, and administrative security measures to protect personal data. source
- Support
- The documentation directs users to Slack or Discord for questions and offers a team Slack or Teams channel by email. source
Checked against the maker's pages on 7 Oct 2026.