Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

TensorZero

tensorzero.com

  • Free plan
  • Linux
  • Web
  • API
  • Self-hosted
7.8 Club score Visit tensorzero.com

What TensorZero is

TensorZero is an open-source stack for building and operating LLM applications. It combines a gateway with observability, optimization, evaluation, and experimentation. The gateway offers a unified API for documented model providers including Anthropic, AWS Bedrock, Azure, Google AI Studio, OpenAI, OpenRouter, vLLM, and xAI, as well as OpenAI-compatible APIs. It supports routing, retries, fallbacks, prompt templates, schemas, multi-step workflows, and A/B tests. Inference and feedback data go to a database you configure and can query with SQL; feedback can be tied to individual inferences or episodes. Deployment options include Docker, Docker Compose, Kubernetes with Helm, or building from source. API key authentication can protect gateway endpoints other than status and health checks. The gateway also collects pseudonymous aggregate usage analytics, which it says exclude application data and inference inputs and outputs; collection can be disabled. TensorZero is free and self-hosted under the Apache 2.0 License, but its homepage says the project is no longer maintained, and GitHub marks the repository archived and read-only.

Who it suits

TensorZero may suit teams building LLM applications that want configurable routing and SQL access to inference and feedback data. It is less suitable for teams that need a maintained project, since its homepage says development has stopped.

Good to know, and what to watch

  • Unified API for documented providers and compatible endpoints
  • Supports fallbacks, retries, and traffic routing
  • Stores inference and feedback data in your database
  • Deployable with Docker, Helm, or from source
  • No longer maintained; GitHub repository is archived
  • Requires a self-hosted deployment
  • Usage analytics are collected by default, though configurable

TensorZero LLMOps platform

Free

Free plan

The verdict

TensorZero brings LLM routing, observability, and experimentation into a self-hosted stack with data stored in your database. Its archived, unmaintained status is a significant consideration for teams evaluating it.

What the maker publishes

Clients
TensorZero supports the OpenAI SDK, OpenTelemetry, and clients for major programming languages. source
Company
The FAQ describes the team as based in New York City and names Viraj Mehta and Gabriel Bianconi as its founders. source
Current status
The homepage says TensorZero remains available on GitHub but is no longer maintained. source
Data control
TensorZero documentation says it stores its data in the user's own database, which can be queried directly with SQL. source
Deployment
The gateway and UI can be deployed with Docker, Docker Compose, or Kubernetes and Helm, or built from source. source
Deployment
The gateway can be deployed with Docker, Docker Compose, Kubernetes using a reference Helm chart, or built from source with Cargo. source
Experiments and workflows
The gateway supports multi-step workflows and traffic routing for A/B tests, with feedback assignable to individual inferences or episodes. source
Features
The product includes routing, retries, fallbacks, prompt templates, schemas, and A/B testing. source
Gateway
The gateway provides a unified API for major LLM providers, with fallbacks and reported sub-millisecond P99 latency overhead under extreme load. source
Gateway
The gateway provides a unified API for accessing supported LLM providers. source
Integrations
Documented providers include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, and OpenAI-compatible endpoints. source
License
The FAQ describes TensorZero as open source under the Apache 2.0 License. source
Maintenance status
The official site says TensorZero remains available on GitHub but is no longer maintained. source
Model integrations
Documented providers include Anthropic, AWS Bedrock, Azure, Google AI Studio, OpenAI, OpenRouter, vLLM, and xAI; OpenAI-compatible APIs are also supported. source
Notable limitation
The FAQ says TensorZero had no revenue model at the time described and planned a managed service for the future; the current homepage says the project is no longer maintained. source
Observability
The gateway stores inference and feedback data in the configured database for observability, experimentation, and optimization. source
Observability
TensorZero can store inference and feedback data for observability, optimization, evaluations, and experimentation in Postgres or ClickHouse. source
Privacy
The gateway documentation says its pseudonymous usage analytics do not include application data or inference inputs and outputs, and can be disabled. source
Product
TensorZero is described in its documentation as an open-source stack combining an LLM gateway, observability, optimization, evaluation, and experimentation. source
Product
TensorZero is an open-source platform that combines an LLM gateway, observability, evaluation, optimization, and experimentation. source
Project status
GitHub marks the TensorZero repository as archived and read-only, archived on June 12, 2026. source
Security
The gateway supports API key authentication, which can be enabled to protect its endpoints other than status and health checks. source
Security
TensorZero's privacy policy says it uses physical, technical, organizational, and administrative security measures to protect personal data. source
Support
The documentation directs users to Slack or Discord for questions and offers a team Slack or Teams channel by email. source

Checked against the maker's pages on 7 Oct 2026.