VRUNAI
vrunai.com
- Free plan
- Web
What VRUNAI is
VRUNAI evaluates LLM agents across execution paths, tool calls, and final outcomes. Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code, then run scenarios repeatedly to score path consistency. The tool calculates cost per scenario and provider from token usage and includes model pricing and context-window statistics. Listed providers include OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers. VRUNAI offers a terminal interface and a React web app that can run locally or in a hosted browser version. Local inference works through Ollama and custom API endpoints, including OpenAI-compatible APIs. The CLI runs on the user’s machine; web-app keys stay in browser localStorage and go directly to configured provider APIs. The project states it has no accounts, backend, or telemetry, is fully client-side, and is AGPL-3.0 licensed. VRUNAI itself has no usage limits or fees, but provider API calls may cost money.
Who it suits
It suits people evaluating LLM agents and comparing behavior across providers. Teams or individuals wanting YAML-defined scenarios, repeatability scoring, and provider-cost calculations can run it locally or in the browser.
Good to know, and what to watch
- YAML scenarios without writing code
- Repeated runs measure path consistency
- Tracks token-based scenario costs
- Supports local Ollama inference
- No accounts, backend, or telemetry
- Configured providers may charge API fees
- AGPL-3.0 licensing applies
- No trace-ingestion feature is listed
- No safety-evaluation feature is listed
Plans
Every plan and priceFree
Free plan
The verdict
VRUNAI provides client-side agent evaluation with scenario repetition, tool-call checks, and cost visibility. It is appropriate when users can manage provider API access and work within AGPL-3.0 terms.
What the maker publishes
- Agent definitions
- Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code. source
- Backend and telemetry
- The site states that VRUNAI has no accounts, backend, or telemetry and is fully client-side. source
- Consistency
- It can run each scenario multiple times and score how consistently the agent follows the same path. source
- Cost tracking
- It calculates cost per scenario and provider from token usage and includes model pricing and context window statistics. source
- Intended users
- The site presents VRUNAI for people evaluating LLM agents and comparing their behavior across providers. source
- Interfaces
- VRUNAI offers a terminal interface and a React web app, which can run locally or be tried in a hosted browser version. source
- License
- VRUNAI is open source under the AGPL-3.0 license. source
- Local model support
- The FAQ says it supports local inference through Ollama and custom API endpoints, including OpenAI-compatible APIs. source
- Notable cost limit
- VRUNAI itself has no usage limits, but users pay for LLM API calls to the providers they configure. source
- Privacy
- The CLI runs on the user's machine and the site says its keys stay in the terminal; web app keys are stored in browser localStorage and sent directly to configured provider APIs. source
- Providers
- The site lists OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers. source
- Support
- The site links to GitHub for bug reports and feature requests. source
- What it does
- VRUNAI evaluates LLM agents across execution path, tool calls, and final outcomes. source
Checked against the maker's pages on 7 Oct 2026.