Future AGI AI Evaluation SDK
futureagi.com
- Free plan
- Open source
- Windows
- macOS
- Linux
- Web
- API
- Self-hosted
What Future AGI AI Evaluation SDK is
Future AGI AI Evaluation SDK is an open-source platform for evaluating, optimizing, monitoring, and protecting AI agents. Its evaluation options include heuristic, code, LLM-as-judge, and agentic methods, with templates and CI/CD support. The traceAI instrumentation converts LLM calls, tool use, retrieval, and chain steps into OpenTelemetry spans. Simulation supports text and voice testing with personas and scenarios; the free plan includes 1M text simulation tokens and 60 voice minutes each month.
The platform lists integrations and SDK packages for Python, TypeScript, and Java, including connections to OpenAI, Anthropic, Google GenAI, Vertex AI, AWS Bedrock, and Azure OpenAI. Its Apache 2.0 offering includes Docker and SDK self-hosting options. Future AGI says a Docker Compose deployment keeps traces, datasets, evaluations, and model calls within the customer's network. Free usage pauses at its cap, while pay-as-you-go usage can incur overage charges. The product is presented for startup and enterprise teams building and operating AI agents.
Who it suits
Future AGI suits startup and enterprise teams building or operating AI agents who need evaluation, tracing, simulation, or self-hosting options. Teams using the Free plan should account for its cap, which pauses usage when reached.
Good to know, and what to watch
- Includes heuristic, code, LLM-as-judge, and agentic evaluations
- Provides CI/CD support and evaluation templates
- TraceAI instrumentation creates OpenTelemetry spans
- Free plan includes text and voice simulation
- Offers Docker and SDK self-hosting options
- Free-plan usage pauses when it reaches its cap
- Pay-as-you-go usage can incur overage charges
- ISO 42001 is in progress
Plans
Every plan and priceFree
Pay-as-you-go
Boost
Scale
Free plan
Getting started
- Visit https://futureagi.com/.
- Choose the web or API route, or use a listed Linux, macOS, or Windows option.
- For self-hosting, use the Docker, Python SDK, or Node SDK option.
- Start with the Free plan or select a paid plan.
- Use the listed integrations and SDK packages to connect supported services.
Limits to check
The Free plan includes 30-day data retention and monthly limits of 2K AI credits, 100K gateway requests, 100K cache hits, 1M text simulation tokens, and 60 voice minutes. Free usage pauses at its cap; Pay-as-you-go continues beyond the free tier and incurs overage charges.
The verdict
Future AGI brings evaluation, tracing, and simulation tools together for AI agent work, with both hosted and self-hosted options described. Check the plan limits and usage behavior against the workloads you expect to run.
Questions people ask
Is there a free plan?
Yes. The Free plan is $0/month and includes 50 GB storage, 2K AI credits, 100K gateway requests, 100K cache hits, 1M text simulation tokens, 60 voice minutes, and 30-day data retention.
What do paid plans cost?
Pay-as-you-go starts at $0/month plus usage. Boost is $250/month, Scale is $750/month, and Enterprise is $2,000/month.
Is the platform open source?
The homepage describes it as Apache 2.0 licensed and lists Docker, Python SDK, and Node SDK self-hosting options.
Which platforms are available?
Listed options are API, Linux, macOS, self-hosted, web, and Windows.
Which integrations and SDK languages are listed?
The integrations page lists 79 integrations and SDK packages for Python, TypeScript, and Java. Featured providers include OpenAI, Anthropic, Google GenAI, Vertex AI, AWS Bedrock, and Azure OpenAI.
What happens when Free usage reaches its cap?
Usage pauses at the Free-plan cap. Pay-as-you-go usage continues and incurs overage charges.
What the maker publishes
- API scope
- The API reference covers simulations, evaluations, scenarios, personas, datasets, annotations, test runs and execution analytics. source
- Audience
- The site presents the product for startups and enterprise teams building and operating AI agents. source
- Built-in evaluators
- The documentation lists 132 ready-made evaluators covering quality, safety, retrieval, format and related categories. source
- Company
- Future AGI Inc. is privately held and headquartered in San Francisco. source
- Documentation depth
- The docs include Python SDK references, a complete REST API reference, integration guides, cookbooks and an MCP server. source
- Documentation limitation
- Future AGI describes its documentation as a work in progress and asks readers to submit feedback on pages. source
- Enterprise deployment
- The enterprise page offers managed cloud, private cloud in the customer's AWS, GCP, or Azure account, and air-gapped on-premise deployment. source
- Evaluation
- Its evaluation product includes heuristic, code, LLM-as-judge, and agentic evaluations, with templates and CI/CD support. source
- Evaluation layer
- The evaluation system scores AI responses against user-defined quality goals and supports text and voice inputs. source
- Founded and size
- LinkedIn lists Future AGI as founded in 2024 with 11–50 employees. source
- Free-plan limit
- The pricing FAQ says free-plan usage pauses at its cap, while pay-as-you-go usage continues and incurs overage charges. source
- Funding
- Preqin reports a $1.6 million February 2025 pre-seed round co-led by Powerhouse Ventures and Snow Leopard Ventures. source
- Headquarters
- San Francisco, California, United States; Bengaluru, Karnataka, India source
- Integrations
- The integrations page lists 79 integrations and SDK packages for Python, TypeScript, and Java. source
- Integrations
- Future AGI documents integrations with OpenAI, LangChain, LlamaIndex and more than 30 frameworks. source
- License
- The public platform repository identifies the core as Apache License 2.0 and self-hostable. source
- Limit: deployment maturity
- Another independent review cites nightly-release warnings, rough edges and no official Kubernetes or Helm support at its review date. source
- Limit: Gartner complaints
- The Gartner review page lists UI and console issues, a security gap and performance impact among dislikes. source
- Limit: integrations and exports
- G2 reviewers ask for more BI exports and annotation-tool integrations. source
- Limit: self-host operations
- An independent review reports that self-hosting can involve roughly 17 Docker containers and substantial ClickHouse/Kafka operations work. source
- Limit: setup and usability
- G2 reviewers report trial-and-error during setup, unclear initial documentation and reduced intuitiveness for nontechnical users. source
- Limit: tracing coverage
- A G2 reviewer says some classic machine-learning models could not be traced. source
- Local metrics
- The SDK documentation says more than 76 local metrics run without a Future AGI API key. source
- Notable news: self-hosting
- The October 13, 2026 changelog announced a signed Helm chart and a standalone self-host deployment of about three containers using roughly 1 GB idle memory. source
Checked against the maker's pages on 15 Oct 2026.