OpenZiti LLM Gateway

6.6easy start · #25of 33
in LLM Gateway Software

Well documentedyes
Runs where you workyes

Runs on api, Linux, Mac, self-hosted, Windows.

OpenZiti LLM Gateway is ranked #25 of 33 in LLM gateway software on The Geeks Club. It runs on API, Linux, macOS, Self-hosted, Windows.

Compared on LLM gateway software

Fallback routing: Yes
Usage analytics: Yes
Self-hosted deployment: Yes
Budget controls: Yes
Virtual API keys: Yes

Best OpenZiti LLM Gateway alternatives

See all 20

Bifrost Lesson 17.8 Kong Gateway Lesson 27.8 LLM Gateway Lesson 37.6 Requesty Lesson 47.6 TrueFoundry AI Gateway Lesson 57.6 Helicone Lesson 67.5

The record

Purpose: OpenZiti LLM Gateway is an OpenAI-compatible API proxy that routes requests to OpenAI, Anthropic, and OpenAI-compatible backends.
Routing: Optional semantic routing uses keyword heuristics, embedding similarity, and an LLM classifier to choose a model when the request omits one.
Provider integrations: The project documents routing to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang, and other OpenAI-compatible backends.
Anthropic support: The gateway translates between OpenAI format and Anthropic’s Messages API.
Streaming: Chat completions support Server-Sent Events streaming across providers.
Load balancing: It supports weighted round-robin distribution, health checks, and automatic failover across inference endpoints.
Network security: The project describes using zrok over OpenZiti to connect gateway and backend across NAT or network boundaries without firewall rules or port forwarding.
Client authentication: Virtual API keys can authenticate clients and be restricted to models with glob patterns; the health and metrics endpoints remain unauthenticated.
Observability: When enabled, Prometheus-exported metrics include request counts, latency, token counts, routing decisions, provider errors, and endpoint health.
Tracing: Optional tracing logs request metadata and truncated message content, with a configurable maximum content length.
Deployment: The project describes a single Go binary configured with one YAML file and says it needs no database, message queue, or sidecar.
Platforms: Pre-built binaries are available for Linux, macOS, and Windows, and the gateway can also be installed or built with Go.
License: The repository is licensed under Apache 2.0.
Intended users: The README describes solving connectivity and routing for gateways, GPU servers behind NAT, and clients that need access without opening ports.

Where it ranks on The Geeks Club

Best LLM Gateway Software in 2026#25 of 33

Is OpenZiti LLM Gateway yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote OpenZiti LLM Gateway

Sources

github.com/openziti/llm-gateway· checked 9 Oct 2026