ChainForge
chainforge.ai
- Free plan
- Web
- Self-hosted
What ChainForge is
ChainForge is an open-source visual environment for prompt engineering, LLM evaluation, and experimentation. It can query multiple language models at once, then compare responses across prompt variations, models, and settings. Users can score responses with code-based or LLM-based evaluators and plot results. Chat Turn nodes support multi-turn experiments by continuing conversations, templating chat messages, and passing a conversation from one model to another. ChainForge imports JSONL, XLSX, and CSV files and exports inspected response data as XLSX. Its listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama. A local installation can add custom providers written as Python completion functions. The web version has a limited feature set; local use supports API keys from environment variables, Python evaluation code, and locally hosted Alpaca or Llama models via Dalai. The project is an open beta in active development. Its documentation says web users must set API keys each time they load the app, and that forgotten encryption passwords cannot be recovered.
Who it suits
ChainForge suits people comparing prompts and language models or evaluating responses with code or model-based scorers. The local version fits users who need Python evaluation code, environment-based keys, or locally hosted models.
Good to know, and what to watch
- Queries multiple models and prompt variations at once.
- Supports code-based and LLM-based response scorers.
- Imports JSONL, XLSX, and CSV files.
- Local installs can add Python completion-function providers.
- Web users can share an experiment with a unique link.
- The web version has a limited feature set.
- Web users must set API keys each time they load it.
- Forgotten encryption passwords prevent access to saved flows or settings.
- The beta supports Chrome, Firefox, Edge, and Brave only.
Plans
Every plan and priceFree and open-source
Open source
Getting started
- Open https://chainforge.ai/
- Use the web version or install ChainForge for self-hosting
- Run it in Chrome, Firefox, Edge, or Brave
- Set provider API keys when loading the web version
- Configure providers, prompts, and evaluation nodes
- Import JSONL, XLSX, or CSV data for experiments
Limits to check
The web version exposes fewer features than the local installation. Local encryption protects settings or saved files, but forgetting its password prevents access.
The verdict
ChainForge provides visual tools for comparing model responses and evaluating prompt experiments, with a broader feature set available locally. Its open beta status, web-version limits, and unrecoverable encryption passwords are relevant considerations.
Questions people ask
Is ChainForge free?
Yes. The listed Free and open-source plan costs nothing.
Which platforms does ChainForge support?
It is available as a web application or self-hosted installation.
Which model providers are supported?
The supported list includes OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama.
Does ChainForge store API keys?
Its documentation says API keys are not stored in cookies, localStorage, or the server. Web users must enter them each time the application loads.
Can ChainForge use custom providers?
A locally installed version can load custom providers written as Python completion functions.
What the maker publishes
- API key handling
- The documentation says the web version does not store API keys in cookies, localStorage or the server, so users must set them each time they load ChainForge. source
- API keys
- The documentation says ChainForge does not store API keys and that users must set them each time they load the web version. source
- Browser support
- The installation guide lists Chrome, Firefox, Edge and Brave, and says the beta does not support other browsers. source
- Compare models and prompts
- It can query multiple LLMs at once and compare response quality across prompt variations, models and model settings. source
- Compare prompts and models
- It can query multiple LLMs at once and compare responses across prompt variations, models and model settings. source
- Conversation testing
- Chat Turn nodes can continue multiple conversations, template chat messages and pass a conversation from one LLM to another. source
- Creator
- The documentation says ChainForge was created by Ian Arawjo, who serves as its chief developer and maintainer. source
- Custom integrations
- Locally installed ChainForge can add providers through Python completion functions loaded in the Custom Providers settings tab. source
- Custom providers
- A locally installed version can call custom providers written as Python completion functions. source
- Data and exports
- ChainForge can import JSONL, XLSX and CSV spreadsheets and export inspected response data as XLSX. source
- Encryption limitation
- The documentation warns that encrypted flows and settings cannot be accessed if the user forgets the password. source
- Evaluate responses
- Users can evaluate responses with code-based or LLM-based scorers and plot evaluation results. source
- Known security limitation
- The documentation says forgetting the encryption password prevents access to saved flows or settings, and passwords are stored in hashed form and cannot be recovered. source
- Local encryption
- When running locally, users can enable encryption for settings or all locally saved files with the secure-mode command options. source
- Local features
- Local installation supports API keys from environment variables, Python evaluator nodes and Ollama-hosted models. source
- Project status
- ChainForge is described as being in active development and provided as an open beta test. source
- Providers
- The supported-models page lists OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama. source
- Purpose
- ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation. source
- Support
- The homepage directs users to submit GitHub issues or a Google Form for questions, comments or feature requests. source
- Web sharing
- The web version has a Share button that creates a unique link to an LLM experiment. source
- Web version limit
- The web version has a limited feature set; the full local version supports loading API keys from environment variables, Python evaluation code and locally hosted Alpaca or Llama models via Dalai. source
- What it does
- ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation. source
Checked against the maker's pages on 8 Oct 2026.