Memoturn is an open-source platform for engineering applications built with large language models. It brings tracing, cost and latency metrics, evaluations, prompt management, monitors, and datasets together. Traces, spans, generations, and scores appear on a waterfall timeline with session views and per-user rollups. Teams can run dataset experiments offline, evaluate sampled production traces, or route examples to human reviewers for scoring. Its prompt registry manages versions across production, staging, and custom channels, with experiments that split live traffic between versions. Memoturn accepts OpenTelemetry data through OTLP/JSON and GenAI semantic conventions. It lists TypeScript, Python, and Go SDKs, alongside a REST API and OpenTelemetry receiver. Integrations include model providers, frameworks, and vector stores, though some first-party wrappers are limited to TypeScript or Python. The Apache-2.0 core is self-hostable, with telemetry kept on infrastructure the operator owns. Deployment options include local use, Docker Compose, or Helm on Kubernetes; telemetry storage can use Apache Doris or PostgreSQL, with PostgreSQL described for smaller installs. The maker lists no paid tier or usage limits on the opened pages.
Who it is for
Memoturn suits teams building LLM applications that need to inspect production traffic, monitor spend, detect quality regressions, and manage prompts. It is relevant to teams wanting self-hosted deployment and telemetry on infrastructure they own.
What is good
- Combines traces, evaluations, prompts, and metrics.
- Supports offline datasets and production trace evaluation.
- Includes TypeScript, Python, and Go SDKs.
- Can be self-hosted with Doris or PostgreSQL storage.
What to know first
- Some first-party integration wrappers are TypeScript- or Python-only.
- Go coverage includes OTel export helpers and manual tracing.
Verdict
Memoturn combines observability, evaluation, and prompt workflows with self-hosting options. Review integration coverage for your language and choose a supported deployment and storage setup.
Memoturn plans and pricing
All plansCompared on model monitoring software
- Performance monitoring
- Yes
- Deployment options
- both
