Experimently
getexperimently.com
- Free plan
- Open source
- Windows
- macOS
- Linux
- Web
- iPhone and iPad
- Android
- API
- Self-hosted
What Experimently is
Experimently is an open-source experimentation platform for A/B testing and feature flags that runs on infrastructure you manage. Its Free Preview costs nothing when self-hosted on your AWS account, and the maker says all features are included; there is no hosted tier. Experimently supports A/B and multivariate testing, sequential tests, CUPED, multi-armed bandits, Bayesian testing, split URL tests and LLM/model evaluation. Rollout controls include sticky assignment, staged rollouts, mutual-exclusion groups, a global holdout and automatic rollback based on error-rate and latency thresholds. Targeting offers more than 20 operators, including semantic version comparison, geo-distance, time windows, JSON paths and array operations. Analysis includes frequentist and Bayesian methods, sequential mSPRT testing and segment breakdowns with Bonferroni correction. SDKs are listed for many languages and mobile platforms, but the maker says they are not yet on package registries; installation instructions are in each SDK guide. Deployment options include Docker Compose, Kubernetes with Helm, or an AWS CDK app. Assignment data stays on the user's infrastructure.
Who it suits
Experimently suits teams that want feature flags and experimentation while keeping assignment data on infrastructure they manage. It requires a self-hosted deployment rather than a hosted service.
Good to know, and what to watch
- Free Preview includes all features at no cost.
- Supports frequentist and Bayesian analysis methods.
- Automatic rollback can use error-rate and latency thresholds.
- Data assignment stays on user infrastructure.
- There is no hosted tier.
- SDKs are not yet available through package registries.
- Free Preview requires self-hosting on an AWS account.
Plans
Every plan and priceFree Preview
Free plan
The verdict
Experimently offers a broad set of experiment and rollout controls, but puts deployment and operation on the user. Teams should account for the self-hosted model and SDK installation outside package registries.
What the maker publishes
- A/B and multivariate testing
- The product documentation covers A/B, multivariate, sequential, split-URL, Bayesian and bandit experiments. source
- Compliance reporting
- The documentation provides export functions for SOC 2 and ISO 27001 reports and describes HIPAA-related encryption, retention and BAA features. source
- CUPED
- CUPED variance reduction is documented as a method for reducing variance using pre-experiment covariates. source
- Data handling
- The maker says assignment data stays on the user's own infrastructure and that there is no hosted tier. source
- Data integrations
- Documentation lists AWS, Snowflake, BigQuery, Redshift, Databricks, ClickHouse and MySQL connectors. source
- Deployment
- Users can deploy with Docker Compose, Kubernetes using a Helm chart, or an AWS CDK app targeting ECS Fargate, Aurora PostgreSQL, and ElastiCache in their own AWS account. source
- Deployment options
- Customers can run published containers with Docker Compose, deploy to Kubernetes with Helm, or use an AWS CDK deployment. source
- Documentation depth
- The documentation site includes quick starts, API schemas, SDK guides, integration references, security topics, deployment instructions and statistics references. source
- Experiment controls
- The platform supports sticky assignment, staged rollouts, mutual-exclusion groups, global holdouts and automatic rollback triggered by error-rate or latency thresholds. source
- Experiments and rollouts
- It supports sticky assignment, staged rollouts, mutual-exclusion groups, a global holdout, and automatic rollback based on error-rate and latency thresholds. source
- Independent maintainer
- PyPI identifies amarkanday as the sole maintainer of the placeholder experimently package. source
- Independent package maturity
- PyPI classifies the package as development status 1, Planning, and lists Python 3.9 or newer as a requirement. source
- Independent performance target
- The project article sets a target of under 50 ms at p99 for feature-flag evaluation and more than 10,000 assignments per second. source
- Independent planned architecture
- The project article describes an AWS architecture using ECS/Fargate, Lambda, Aurora PostgreSQL, DynamoDB, ElastiCache, S3, Kinesis, EventBridge, OpenSearch, Glue and Athena. source
- Independent project date
- The product-management article describing the platform was published March 28, 2025. source
- Independent project origin
- Ashish Markanday wrote that building the platform was a personal project intended to create a production-quality experimentation system with AI assistance in roughly three to four months. source
- Independent risk: complexity
- The project article identifies high-throughput real-time evaluation, data integrity, integration complexity and statistical validity as material implementation risks. source
- Integration API
- Integration endpoints support Jira, Salesforce and GitHub, including webhook receivers and role-based permissions. source
- Integrations
- The documentation lists connections for Snowflake, BigQuery, Redshift, Databricks, ClickHouse, MySQL, Jira, Salesforce, GitHub, Slack, and email. source
- JavaScript SDK
- The JavaScript client supports browser and Node.js use, variant assignment, feature-flag evaluation and event tracking. source
- Licence
- The core platform is Apache-2.0 licensed; the SDKs are MIT licensed. source
- License and price
- The platform is Apache-2.0 licensed, and the maker says its build profiles have no price; its Free Preview is listed at $0/month. source
- Limit: hosting responsibility
- There is no hosted tier, so users must provision and operate the deployment in their own infrastructure or AWS account. source
- Limit: SDK registry availability
- The homepage says JavaScript and Python SDKs are beta releases while the other listed SDKs are not yet published in package registries and must be installed from the repository. source
Checked against the maker's pages on 14 Oct 2026.