Mutatest
mutatest.readthedocs.io
- Free plan
- Open source
- Windows
- macOS
- Linux
- API
- Self-hosted
What Mutatest is
Mutatest is a free command-line diagnostic tool for Python projects that checks whether tests catch small code changes, known as mutants. It uses Python's Abstract Syntax Tree grammar to generate valid mutations and modifies local __pycache__ files rather than source files. If coverage data is available, it can restrict mutation targets to lines marked as covered. Mutation locations and candidate changes are sampled randomly, and a seed can make a trial reproducible. Command-line controls include run modes, mutation categories, file exclusions, timeouts, reports and survivor thresholds; an API supports custom mutation controls. The documentation describes using commands such as pytest and coverage data to filter targets. Mutatest can raise an exception when surviving mutants meet a configured threshold, supporting automated runs such as a CI stage. The project says it was tested on Linux, Windows and macOS. Installation requires Python 3.7 or 3.8; multiprocessing is supported on 3.8, but not parallelization on 3.7. Repeated test-suite runs can be slow, so the documentation positions it as a diagnostic rather than a check for every commit.
Who it suits
Mutatest suits Python developers who want to assess whether their tests detect small code changes, including teams adding a mutation check to CI. It is less suited to running the full test suite on every commit.
Good to know, and what to watch
- Can restrict targets to lines marked covered.
- Random seeds make trials reproducible.
- Can set a threshold for surviving mutants.
- Changes cache files rather than source files.
- Free and open source under the MIT License.
- Installation requires Python 3.7 or 3.8.
- Only pytest has been tested to date.
- Parallelization is unsupported on Python 3.7.
- Repeated test-suite runs can be slow.
Plans
Every plan and priceMutatest
Free plan
Getting started
- Open the Mutatest documentation website
- Use Python 3.7 or Python 3.8
- Install and run the command-line tool on Linux, macOS, or Windows
- Provide a pytest test command and optional coverage data
- Choose mutation controls, exclusions, timeouts, reports, and survivor thresholds
Limits to check
Mutatest is free with no paid tiers. Repeated test-suite execution can be slow; multiprocessing works on Python 3.8 but not Python 3.7.
The verdict
Mutatest offers configurable mutation checks for Python test suites, including survivor thresholds for automated runs. Its documented Python versions, pytest testing limitation and runtime warning should factor into adoption.
Questions people ask
What does Mutatest test?
It checks whether a Python test suite detects small, generated code changes called mutants.
Which languages and frameworks are supported?
The supported language is Python, and pytest is the documented test framework.
Can Mutatest use coverage data?
Yes. Available coverage data can restrict mutation targets to covered lines.
Can it run in continuous integration?
Yes. It can raise an exception when surviving mutants reach a configured threshold.
How much does Mutatest cost?
The Mutatest plan is free, and the project is free and open source under the MIT License.
What the maker publishes
- API
- Mutatest includes an API for custom mutation controls. source
- API objects
- The documented API centers on Genome, GenomeGroup and Mutant objects. source
- AST mutation engine
- Mutations are generated from Python's Abstract Syntax Tree grammar. source
- CI use
- Mutatest can raise an exception when surviving mutants meet a configured threshold, for use in automated runs such as a continuous integration stage. source
- CLI configuration
- Settings can be supplied through mutatest.ini or setup.cfg sections named mutatest or tool:mutatest. source
- Controls
- The command line supports run modes, mutation category selection, file exclusions, timeouts, output reports, and survivor thresholds. source
- Coverage
- When coverage data is available, mutation targets can be restricted to lines marked as covered. source
- Coverage guidance
- Coverage data can restrict mutation targets to code executed by tests. source
- Custom API
- The package exposes an API for custom mutation controls. source
- Diagnostic positioning
- The installation guide describes Mutatest as a diagnostic tool and cautions that running a full suite repeatedly can be slow for continuous integration. source
- Documentation depth
- Documentation includes installation, command-line controls, API reference, API tutorial, mutation operators and contributing guidance. source
- Execution modes
- The command line supports fast and detection or survival stopping modes, including f, s, d and sd. source
- Independent activity record
- Ecosyste.ms reported the last repository push as February 17, 2023. source
- Independent popularity record
- Ecosyste.ms reported 97 stars, 18 forks and two watchers when it last synced the repository. source
- Independent repository record
- Ecosyste.ms records the repository as created on December 22, 2018, with MIT licensing. source
- Install channels
- Installation is documented through pip, conda-forge or a local source checkout. source
- Integrations
- The documentation describes running test commands such as pytest and using coverage data to filter mutation targets. source
- Known limitation
- The documentation warns that repeated test-suite runs can make mutation testing slow, so it is intended as a diagnostic tool rather than a run over an entire codebase on every commit. source
- Latest release
- PyPI lists version 3.1.0 as released on February 20, 2022. source
- Licence
- PyPI identifies the project as MIT licensed and OSI-approved. source
- Limit: diagnostics
- The public issue tracker contains an open report about being unable to retrieve failing-test information for mutants, opened May 28, 2025. source
- Limit: maintenance
- The public issue tracker contains an open request to support newer Python versions, opened November 30, 2025. source
- Limit: mutation coverage
- An open issue reports that type hints are not ignored as mutation targets. source
- Limit: parallel integration
- An independent Reddit comparison says Mutatest lacks pytest-xdist integration and that its last PyPI release was in 2022. source
Checked against the maker's pages on 15 Oct 2026.