Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

VRUNAI

vrunai.com

  • Free plan
  • Web
7.2 Club score Visit vrunai.com

What VRUNAI is

VRUNAI evaluates LLM agents across execution paths, tool calls, and final outcomes. Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code, then run scenarios repeatedly to score path consistency. The tool calculates cost per scenario and provider from token usage and includes model pricing and context-window statistics. Listed providers include OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers. VRUNAI offers a terminal interface and a React web app that can run locally or in a hosted browser version. Local inference works through Ollama and custom API endpoints, including OpenAI-compatible APIs. The CLI runs on the user’s machine; web-app keys stay in browser localStorage and go directly to configured provider APIs. The project states it has no accounts, backend, or telemetry, is fully client-side, and is AGPL-3.0 licensed. VRUNAI itself has no usage limits or fees, but provider API calls may cost money.

Who it suits

It suits people evaluating LLM agents and comparing behavior across providers. Teams or individuals wanting YAML-defined scenarios, repeatability scoring, and provider-cost calculations can run it locally or in the browser.

Good to know, and what to watch

  • YAML scenarios without writing code
  • Repeated runs measure path consistency
  • Tracks token-based scenario costs
  • Supports local Ollama inference
  • No accounts, backend, or telemetry
  • Configured providers may charge API fees
  • AGPL-3.0 licensing applies
  • No trace-ingestion feature is listed
  • No safety-evaluation feature is listed

Free

Free

Free plan

The verdict

VRUNAI provides client-side agent evaluation with scenario repetition, tool-call checks, and cost visibility. It is appropriate when users can manage provider API access and work within AGPL-3.0 terms.

What the maker publishes

Agent definitions
Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code. source
Backend and telemetry
The site states that VRUNAI has no accounts, backend, or telemetry and is fully client-side. source
Consistency
It can run each scenario multiple times and score how consistently the agent follows the same path. source
Cost tracking
It calculates cost per scenario and provider from token usage and includes model pricing and context window statistics. source
Intended users
The site presents VRUNAI for people evaluating LLM agents and comparing their behavior across providers. source
Interfaces
VRUNAI offers a terminal interface and a React web app, which can run locally or be tried in a hosted browser version. source
License
VRUNAI is open source under the AGPL-3.0 license. source
Local model support
The FAQ says it supports local inference through Ollama and custom API endpoints, including OpenAI-compatible APIs. source
Notable cost limit
VRUNAI itself has no usage limits, but users pay for LLM API calls to the providers they configure. source
Privacy
The CLI runs on the user's machine and the site says its keys stay in the terminal; web app keys are stored in browser localStorage and sent directly to configured provider APIs. source
Providers
The site lists OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers. source
Support
The site links to GitHub for bug reports and feature requests. source
What it does
VRUNAI evaluates LLM agents across execution path, tool calls, and final outcomes. source

Checked against the maker's pages on 7 Oct 2026.