PromptMetrics
github.com
- Free plan
- Windows
- macOS
- Linux
- Web
- API
- Self-hosted
What PromptMetrics is
PromptMetrics is a free, MIT-licensed, self-hosted prompt registry for versioning prompts and observing LLM activity. Prompts can be stored, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. The platform logs model, token, latency, cost, and custom-tag metadata, exporting through stdout JSON or OpenTelemetry. It tracks agent traces, spans, workflow runs, and environment-labeled prompt versions. Evaluation suites run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner. A compliance scanner checks prompts for PII, API keys, URLs, and IP addresses, assigning risk scores. The playground proxies calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. Prompt content can use filesystem, GitHub, or S3 storage, while metadata uses SQLite or PostgreSQL. Interfaces include an optional Next.js dashboard, CLI, REST API, and Node.js and Python SDKs. Installation uses npm, Docker Compose, or source builds.
Who it suits
PromptMetrics suits teams that need self-hosted prompt versioning, LLM telemetry, evaluations, and compliance checks. It fits developers using Git-backed storage and API or SDK workflows.
Good to know, and what to watch
- Free and MIT licensed
- Git-backed prompt versioning and rollback
- Tracks cost, latency, tokens, and models
- Dataset evaluations and A/B tests
- Supports multiple storage backends
- Self-hosted deployment required
- Default limit is 300 requests per 60 seconds
- Dashboard is optional
- Security guidance requires HTTPS and backups
The verdict
PromptMetrics combines prompt lifecycle management with observability, evaluations, compliance scanning, and provider proxying. Its self-hosted design offers deployment flexibility but leaves installation and operational safeguards to the user.
What the maker publishes
- Agent telemetry
- It tracks agent traces and spans, workflow runs, and prompt versions tagged with environment labels. source
- Compliance scanning
- Its compliance engine scans prompts for PII, API keys, URLs, and IP addresses and assigns risk scores. source
- Dashboard
- An optional Next.js dashboard provides charts and views for prompts, logs, traces, evaluations, A/B tests, compliance, and the playground. source
- Dashboard key handling
- The dashboard authenticates through a BFF proxy so API keys are not stored in browser localStorage. source
- Deployment
- The README provides installation options using npm, Docker Compose, or source, and describes the product as self-hosted. source
- Evaluations and A/B tests
- Evaluation suites can be run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner. source
- Evaluations and testing
- It supports evaluation suites, dataset-backed evaluation runs, and statistical A/B comparisons with winner promotion. source
- Headquarters
- Berlin, Germany source
- Installation
- The README documents installation through npm, Docker Compose, or building from source. source
- Interfaces and SDKs
- The project provides a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs. source
- License
- The repository identifies the project as MIT licensed. source
- LLM providers
- The playground can proxy chat and completion calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. source
- Logging
- Structured logs can capture model, token, latency, cost, and custom-tag metadata and export to stdout JSON or OpenTelemetry. source
- Maker and company details
- The repository says it is maintained by the PromptMetrics team; it does not state a headquarters or founding date. source
- Observability
- It logs LLM request metadata such as model, tokens, latency, cost, and custom tags, and can export data through OpenTelemetry. source
- Prompt versioning
- Prompts can be stored, versioned, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. source
- Purpose
- PromptMetrics is a lightweight, self-hosted prompt registry with Git-backed versioning, metadata logging, and evaluations for LLM observability. source
- Rate limit
- The README lists a default limit of 300 requests per 60-second window per API key, configurable through environment variables. source
- SDKs
- The project lists Node.js and Python SDKs for programmatic access. source
- Security controls
- API keys are HMAC-SHA256 hashed, support scoped permissions and optional expiration, and the dashboard uses a proxy so keys are not stored in browser localStorage. source
- Security guidance
- The security policy recommends HTTPS, a reverse proxy, a long random API key salt, and regular database backups. source
- Storage
- Prompt content is stored in Git, while metadata can use SQLite or PostgreSQL; filesystem, GitHub, and S3 storage drivers are listed. source
- Storage options
- Prompt content can use filesystem, GitHub, or S3 storage, while metadata can use SQLite or PostgreSQL. source
- What it does
- PromptMetrics is a lightweight, self-hosted prompt registry for versioning prompts and tracking LLM activity. source
Checked against the maker's pages on 8 Oct 2026.