Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

ChainForge

chainforge.ai

  • Free plan
  • Web
  • Self-hosted
7.6 Club score Visit chainforge.ai

What ChainForge is

ChainForge is an open-source visual environment for prompt engineering, LLM evaluation, and experimentation. It can query multiple language models at once, then compare responses across prompt variations, models, and settings. Users can score responses with code-based or LLM-based evaluators and plot results. Chat Turn nodes support multi-turn experiments by continuing conversations, templating chat messages, and passing a conversation from one model to another. ChainForge imports JSONL, XLSX, and CSV files and exports inspected response data as XLSX. Its listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama. A local installation can add custom providers written as Python completion functions. The web version has a limited feature set; local use supports API keys from environment variables, Python evaluation code, and locally hosted Alpaca or Llama models via Dalai. The project is an open beta in active development. Its documentation says web users must set API keys each time they load the app, and that forgotten encryption passwords cannot be recovered.

Who it suits

ChainForge suits people comparing prompts and language models or evaluating responses with code or model-based scorers. The local version fits users who need Python evaluation code, environment-based keys, or locally hosted models.

Good to know, and what to watch

  • Queries multiple models and prompt variations at once.
  • Supports code-based and LLM-based response scorers.
  • Imports JSONL, XLSX, and CSV files.
  • Local installs can add Python completion-function providers.
  • Web users can share an experiment with a unique link.
  • The web version has a limited feature set.
  • Web users must set API keys each time they load it.
  • Forgotten encryption passwords prevent access to saved flows or settings.
  • The beta supports Chrome, Firefox, Edge, and Brave only.

Free and open-source

Free

Open source

Getting started

  1. Open https://chainforge.ai/
  2. Use the web version or install ChainForge for self-hosting
  3. Run it in Chrome, Firefox, Edge, or Brave
  4. Set provider API keys when loading the web version
  5. Configure providers, prompts, and evaluation nodes
  6. Import JSONL, XLSX, or CSV data for experiments

Limits to check

The web version exposes fewer features than the local installation. Local encryption protects settings or saved files, but forgetting its password prevents access.

The verdict

ChainForge provides visual tools for comparing model responses and evaluating prompt experiments, with a broader feature set available locally. Its open beta status, web-version limits, and unrecoverable encryption passwords are relevant considerations.

Questions people ask

Is ChainForge free?

Yes. The listed Free and open-source plan costs nothing.

Which platforms does ChainForge support?

It is available as a web application or self-hosted installation.

Which model providers are supported?

The supported list includes OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama.

Does ChainForge store API keys?

Its documentation says API keys are not stored in cookies, localStorage, or the server. Web users must enter them each time the application loads.

Can ChainForge use custom providers?

A locally installed version can load custom providers written as Python completion functions.

What the maker publishes

API key handling
The documentation says the web version does not store API keys in cookies, localStorage or the server, so users must set them each time they load ChainForge. source
API keys
The documentation says ChainForge does not store API keys and that users must set them each time they load the web version. source
Browser support
The installation guide lists Chrome, Firefox, Edge and Brave, and says the beta does not support other browsers. source
Compare models and prompts
It can query multiple LLMs at once and compare response quality across prompt variations, models and model settings. source
Compare prompts and models
It can query multiple LLMs at once and compare responses across prompt variations, models and model settings. source
Conversation testing
Chat Turn nodes can continue multiple conversations, template chat messages and pass a conversation from one LLM to another. source
Creator
The documentation says ChainForge was created by Ian Arawjo, who serves as its chief developer and maintainer. source
Custom integrations
Locally installed ChainForge can add providers through Python completion functions loaded in the Custom Providers settings tab. source
Custom providers
A locally installed version can call custom providers written as Python completion functions. source
Data and exports
ChainForge can import JSONL, XLSX and CSV spreadsheets and export inspected response data as XLSX. source
Encryption limitation
The documentation warns that encrypted flows and settings cannot be accessed if the user forgets the password. source
Evaluate responses
Users can evaluate responses with code-based or LLM-based scorers and plot evaluation results. source
Known security limitation
The documentation says forgetting the encryption password prevents access to saved flows or settings, and passwords are stored in hashed form and cannot be recovered. source
Local encryption
When running locally, users can enable encryption for settings or all locally saved files with the secure-mode command options. source
Local features
Local installation supports API keys from environment variables, Python evaluator nodes and Ollama-hosted models. source
Project status
ChainForge is described as being in active development and provided as an open beta test. source
Providers
The supported-models page lists OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama. source
Purpose
ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation. source
Support
The homepage directs users to submit GitHub issues or a Google Form for questions, comments or feature requests. source
Web sharing
The web version has a Share button that creates a unique link to an LLM experiment. source
Web version limit
The web version has a limited feature set; the full local version supports loading API keys from environment variables, Python evaluation code and locally hosted Alpaca or Llama models via Dalai. source
What it does
ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation. source

Checked against the maker's pages on 8 Oct 2026.