Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

Future AGI AI Evaluation SDK

futureagi.com

  • Free plan
  • Open source
  • Windows
  • macOS
  • Linux
  • Web
  • API
  • Self-hosted
8.7 Club score Visit futureagi.com

What Future AGI AI Evaluation SDK is

Future AGI AI Evaluation SDK is an open-source platform for evaluating, optimizing, monitoring, and protecting AI agents. Its evaluation options include heuristic, code, LLM-as-judge, and agentic methods, with templates and CI/CD support. The traceAI instrumentation converts LLM calls, tool use, retrieval, and chain steps into OpenTelemetry spans. Simulation supports text and voice testing with personas and scenarios; the free plan includes 1M text simulation tokens and 60 voice minutes each month.

The platform lists integrations and SDK packages for Python, TypeScript, and Java, including connections to OpenAI, Anthropic, Google GenAI, Vertex AI, AWS Bedrock, and Azure OpenAI. Its Apache 2.0 offering includes Docker and SDK self-hosting options. Future AGI says a Docker Compose deployment keeps traces, datasets, evaluations, and model calls within the customer's network. Free usage pauses at its cap, while pay-as-you-go usage can incur overage charges. The product is presented for startup and enterprise teams building and operating AI agents.

Who it suits

Future AGI suits startup and enterprise teams building or operating AI agents who need evaluation, tracing, simulation, or self-hosting options. Teams using the Free plan should account for its cap, which pauses usage when reached.

Good to know, and what to watch

  • Includes heuristic, code, LLM-as-judge, and agentic evaluations
  • Provides CI/CD support and evaluation templates
  • TraceAI instrumentation creates OpenTelemetry spans
  • Free plan includes text and voice simulation
  • Offers Docker and SDK self-hosting options
  • Free-plan usage pauses when it reaches its cap
  • Pay-as-you-go usage can incur overage charges
  • ISO 42001 is in progress

Free

Free

Pay-as-you-go

Free

Boost

$250/mo

Scale

$750/mo

Free plan

Getting started

  1. Visit https://futureagi.com/.
  2. Choose the web or API route, or use a listed Linux, macOS, or Windows option.
  3. For self-hosting, use the Docker, Python SDK, or Node SDK option.
  4. Start with the Free plan or select a paid plan.
  5. Use the listed integrations and SDK packages to connect supported services.

Limits to check

The Free plan includes 30-day data retention and monthly limits of 2K AI credits, 100K gateway requests, 100K cache hits, 1M text simulation tokens, and 60 voice minutes. Free usage pauses at its cap; Pay-as-you-go continues beyond the free tier and incurs overage charges.

The verdict

Future AGI brings evaluation, tracing, and simulation tools together for AI agent work, with both hosted and self-hosted options described. Check the plan limits and usage behavior against the workloads you expect to run.

Questions people ask

Is there a free plan?

Yes. The Free plan is $0/month and includes 50 GB storage, 2K AI credits, 100K gateway requests, 100K cache hits, 1M text simulation tokens, 60 voice minutes, and 30-day data retention.

What do paid plans cost?

Pay-as-you-go starts at $0/month plus usage. Boost is $250/month, Scale is $750/month, and Enterprise is $2,000/month.

Is the platform open source?

The homepage describes it as Apache 2.0 licensed and lists Docker, Python SDK, and Node SDK self-hosting options.

Which platforms are available?

Listed options are API, Linux, macOS, self-hosted, web, and Windows.

Which integrations and SDK languages are listed?

The integrations page lists 79 integrations and SDK packages for Python, TypeScript, and Java. Featured providers include OpenAI, Anthropic, Google GenAI, Vertex AI, AWS Bedrock, and Azure OpenAI.

What happens when Free usage reaches its cap?

Usage pauses at the Free-plan cap. Pay-as-you-go usage continues and incurs overage charges.

What the maker publishes

API scope
The API reference covers simulations, evaluations, scenarios, personas, datasets, annotations, test runs and execution analytics. source
Audience
The site presents the product for startups and enterprise teams building and operating AI agents. source
Built-in evaluators
The documentation lists 132 ready-made evaluators covering quality, safety, retrieval, format and related categories. source
Company
Future AGI Inc. is privately held and headquartered in San Francisco. source
Documentation depth
The docs include Python SDK references, a complete REST API reference, integration guides, cookbooks and an MCP server. source
Documentation limitation
Future AGI describes its documentation as a work in progress and asks readers to submit feedback on pages. source
Enterprise deployment
The enterprise page offers managed cloud, private cloud in the customer's AWS, GCP, or Azure account, and air-gapped on-premise deployment. source
Evaluation
Its evaluation product includes heuristic, code, LLM-as-judge, and agentic evaluations, with templates and CI/CD support. source
Evaluation layer
The evaluation system scores AI responses against user-defined quality goals and supports text and voice inputs. source
Founded and size
LinkedIn lists Future AGI as founded in 2024 with 11–50 employees. source
Free-plan limit
The pricing FAQ says free-plan usage pauses at its cap, while pay-as-you-go usage continues and incurs overage charges. source
Funding
Preqin reports a $1.6 million February 2025 pre-seed round co-led by Powerhouse Ventures and Snow Leopard Ventures. source
Headquarters
San Francisco, California, United States; Bengaluru, Karnataka, India source
Integrations
The integrations page lists 79 integrations and SDK packages for Python, TypeScript, and Java. source
Integrations
Future AGI documents integrations with OpenAI, LangChain, LlamaIndex and more than 30 frameworks. source
License
The public platform repository identifies the core as Apache License 2.0 and self-hostable. source
Limit: deployment maturity
Another independent review cites nightly-release warnings, rough edges and no official Kubernetes or Helm support at its review date. source
Limit: Gartner complaints
The Gartner review page lists UI and console issues, a security gap and performance impact among dislikes. source
Limit: integrations and exports
G2 reviewers ask for more BI exports and annotation-tool integrations. source
Limit: self-host operations
An independent review reports that self-hosting can involve roughly 17 Docker containers and substantial ClickHouse/Kafka operations work. source
Limit: setup and usability
G2 reviewers report trial-and-error during setup, unclear initial documentation and reduced intuitiveness for nontechnical users. source
Limit: tracing coverage
A G2 reviewer says some classic machine-learning models could not be traced. source
Local metrics
The SDK documentation says more than 76 local metrics run without a Future AGI API key. source
Notable news: self-hosting
The October 13, 2026 changelog announced a signed Helm chart and a standalone self-host deployment of about three containers using roughly 1 GB idle memory. source

Checked against the maker's pages on 15 Oct 2026.