Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

PromptMetrics

github.com

  • Free plan
  • Windows
  • macOS
  • Linux
  • Web
  • API
  • Self-hosted
7.8 Club score Visit github.com

What PromptMetrics is

PromptMetrics is a free, MIT-licensed, self-hosted prompt registry for versioning prompts and observing LLM activity. Prompts can be stored, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. The platform logs model, token, latency, cost, and custom-tag metadata, exporting through stdout JSON or OpenTelemetry. It tracks agent traces, spans, workflow runs, and environment-labeled prompt versions. Evaluation suites run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner. A compliance scanner checks prompts for PII, API keys, URLs, and IP addresses, assigning risk scores. The playground proxies calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. Prompt content can use filesystem, GitHub, or S3 storage, while metadata uses SQLite or PostgreSQL. Interfaces include an optional Next.js dashboard, CLI, REST API, and Node.js and Python SDKs. Installation uses npm, Docker Compose, or source builds.

Who it suits

PromptMetrics suits teams that need self-hosted prompt versioning, LLM telemetry, evaluations, and compliance checks. It fits developers using Git-backed storage and API or SDK workflows.

Good to know, and what to watch

  • Free and MIT licensed
  • Git-backed prompt versioning and rollback
  • Tracks cost, latency, tokens, and models
  • Dataset evaluations and A/B tests
  • Supports multiple storage backends
  • Self-hosted deployment required
  • Default limit is 300 requests per 60 seconds
  • Dashboard is optional
  • Security guidance requires HTTPS and backups

The verdict

PromptMetrics combines prompt lifecycle management with observability, evaluations, compliance scanning, and provider proxying. Its self-hosted design offers deployment flexibility but leaves installation and operational safeguards to the user.

What the maker publishes

Agent telemetry
It tracks agent traces and spans, workflow runs, and prompt versions tagged with environment labels. source
Compliance scanning
Its compliance engine scans prompts for PII, API keys, URLs, and IP addresses and assigns risk scores. source
Dashboard
An optional Next.js dashboard provides charts and views for prompts, logs, traces, evaluations, A/B tests, compliance, and the playground. source
Dashboard key handling
The dashboard authenticates through a BFF proxy so API keys are not stored in browser localStorage. source
Deployment
The README provides installation options using npm, Docker Compose, or source, and describes the product as self-hosted. source
Evaluations and A/B tests
Evaluation suites can be run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner. source
Evaluations and testing
It supports evaluation suites, dataset-backed evaluation runs, and statistical A/B comparisons with winner promotion. source
Headquarters
Berlin, Germany source
Installation
The README documents installation through npm, Docker Compose, or building from source. source
Interfaces and SDKs
The project provides a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs. source
License
The repository identifies the project as MIT licensed. source
LLM providers
The playground can proxy chat and completion calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. source
Logging
Structured logs can capture model, token, latency, cost, and custom-tag metadata and export to stdout JSON or OpenTelemetry. source
Maker and company details
The repository says it is maintained by the PromptMetrics team; it does not state a headquarters or founding date. source
Observability
It logs LLM request metadata such as model, tokens, latency, cost, and custom tags, and can export data through OpenTelemetry. source
Prompt versioning
Prompts can be stored, versioned, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. source
Purpose
PromptMetrics is a lightweight, self-hosted prompt registry with Git-backed versioning, metadata logging, and evaluations for LLM observability. source
Rate limit
The README lists a default limit of 300 requests per 60-second window per API key, configurable through environment variables. source
SDKs
The project lists Node.js and Python SDKs for programmatic access. source
Security controls
API keys are HMAC-SHA256 hashed, support scoped permissions and optional expiration, and the dashboard uses a proxy so keys are not stored in browser localStorage. source
Security guidance
The security policy recommends HTTPS, a reverse proxy, a long random API key salt, and regular database backups. source
Storage
Prompt content is stored in Git, while metadata can use SQLite or PostgreSQL; filesystem, GitHub, and S3 storage drivers are listed. source
Storage options
Prompt content can use filesystem, GitHub, or S3 storage, while metadata can use SQLite or PostgreSQL. source
What it does
PromptMetrics is a lightweight, self-hosted prompt registry for versioning prompts and tracking LLM activity. source

Checked against the maker's pages on 8 Oct 2026.