Promptfoo
Open-source evaluations and red teaming for LLM applications and agents
Promptfoo is a local-first framework for evaluating and red-teaming LLM applications and agents. Its CLI and Node.js library run test matrices, assertions, regression checks, and vulnerability probes locally or in CI, with an optional hosted platform for teams.
Best for agent capabilities
Useful in agent tasks
Overview
When to use Promptfoo
Promptfoo is listed for Observability, Monitoring, Auth workflows. Promptfoo is a local-first framework for evaluating and red-teaming LLM applications and agents. Its CLI and Node.js library run test matrices, assertions, regression checks, and vulnerability probes locally or in CI, with an optional hosted platform for teams. Consider it when an agent needs this capability through a documented service boundary instead of custom infrastructure maintained inside the application.
In 4agent's task model, the product maps to observability and evaluation, authentication and secrets. That makes it a candidate for workflows that research the web, build and deploy software, manage memory and knowledge, automate business workflows.
Integration checklist
- Access: confirm the none authentication flow, credential scope, rotation process, and whether calls can be attributed to a user or service identity.
- Client support: official or documented clients are listed for typescript.
- Automation: OpenAPI is not currently recorded, MCP is not currently recorded, and CLI support is listed as available. Confirm each requirement in the provider's current documentation.
- Operations: test rate limits, timeouts, retry safety, error payloads, logs, data retention, and the current free-tier and paid pricing model with production-like volume.
Sources and verification
Use these primary sources to verify current behavior before making an implementation or purchasing decision. Product capabilities, limits, and prices can change after a 4agent listing is reviewed.