7 evals and observability
Tracing, evaluation and red-teaming so you can see what an agent did and whether it was right.
- Rank 4
Traces LLM calls and agent steps, then runs evals, datasets and prompt versioning on top of them
- score 96
- 35.8k stars
- pushed
- v4.59.0 on
- MIT
Install the Python SDK
pip install langfuseWhen to use it, when to skip it
Why you'd want it
Langfuse instruments an app through Python and JS/TS SDKs or integrations such as the OpenAI SDK, LangChain, LlamaIndex, Vercel AI SDK and Mastra, and records LLM calls, retrieval and agent actions as traces you can inspect. The same platform covers prompt management with server and client caching, datasets for pre-deployment tests, and evaluations via LLM-as-a-judge, code evaluators, user feedback and manual labeling. You can use Langfuse Cloud or self-host it with Docker Compose, on a VM or on Kubernetes via Helm.
Best for
- Teams that want to self-host LLM observability
- Tracing, prompts and evals in one place
- Python and JS/TS stacks
Use it when
You need to see what your LLM app or agent did on each request and want to evaluate and iterate on prompts against the same traces.
Skip it if
You do not want to run a server stack yourself and cannot use a hosted account; self-hosting means running the Docker Compose or Kubernetes deployment. Code under the ee/ directories is not MIT licensed.
- Rank 24
Runs evals and red-team scans against prompts, models and agents from a CLI, with a web results viewer
- score 81
- 26k stars
- pushed
- code-scan-action-0.2.2 on
- MIT
Install with npm
npm install -g promptfoo promptfoo init --example getting-startedWhen to use it, when to skip it
Why you'd want it
Promptfoo is a CLI and library that runs automated evaluations of prompts and models, compares providers such as OpenAI, Anthropic, Azure, Bedrock and Ollama side by side, and shows results in a web viewer. It also does red teaming and vulnerability scanning, and can review pull requests for LLM-related security and compliance issues. Evals run locally, it works with any LLM API or programming language, and it plugs into CI/CD.
Best for
- Regression tests for prompts in CI
- Comparing models side by side
- Red teaming LLM apps before launch
Use it when
You want repeatable, local eval runs and security scans that can gate changes in CI.
Skip it if
You need tracing and monitoring of live production traffic; promptfoo is built around evals, red teaming and code scanning rather than observability.
- Rank 26
Traces LLM and agent apps over OpenTelemetry and runs evals, datasets and experiments in a local UI
- score 80
- 11.8k stars
- pushed
- arize-phoenix-v20.21.4 on
- Elastic License 2.0
Install with pip
pip install arize-phoenix phoenix serveWhen to use it, when to skip it
Why you'd want it
Phoenix collects traces through OpenTelemetry-based OpenInference instrumentation, with integrations for frameworks such as the OpenAI Agents SDK, Claude Agent SDK, LangGraph and Vercel AI SDK across Python, JavaScript, Java and Go. On top of traces it offers LLM-based evals, versioned datasets, experiments, a prompt playground and prompt management. It runs locally with
phoenix serve, in a container or on Kubernetes, and exposes a remote MCP endpoint so Claude Code, Cursor and other MCP clients can query traces, datasets and experiments.Best for
- Local trace debugging during development
- OpenTelemetry-based stacks
- Self-hosted eval and experiment tracking
Use it when
You want an observability and eval UI on your own machine or infrastructure, instrumented with OpenTelemetry rather than a vendor-specific SDK.
Skip it if
You need an OSI open source license or want to offer it as a hosted service; the Elastic License 2.0 forbids that. For a managed platform Arize points to its separate Arize AX product.
- Rank 47
Traces agent runs and LLM calls, scores them with LLM-as-a-judge metrics and monitors production
- score 68
- 22.6k stars
- pushed
- 2.2.100 on
- Apache-2.0
Install the Python SDK
pip install opik opik configureWhen to use it, when to skip it
Why you'd want it
Opik, built by Comet, logs nested traces through a
@trackdecorator in Python, a TypeScript SDK or OpenTelemetry, so multi-step agent runs and tool calls show up as full trace trees. It adds datasets, experiments, LLM-as-a-judge metrics for hallucination, moderation and RAG, a PyTest integration for CI, online evaluation rules, dashboards and an Agent Optimizer SDK. The full platform, backend included, is Apache-2.0 and can be self-hosted with Docker Compose or Helm, or used on Comet's cloud.Best for
- Teams that want a fully Apache-2.0 self-hosted platform
- Agent tracing and evals in one tool
- Running LLM evals in PyTest
Use it when
You want tracing, evaluation and production monitoring under one open source license you can self-host, or want your coding agent to read traces through Opik's MCP server.
Skip it if
You only need lightweight offline tests; self-hosting runs a multi-container stack with databases and caches, and the alternative is a Comet account.
- Rank 52
Unit tests LLM apps and agents in a Pytest-style workflow with LLM-as-a-judge and other metrics
- score 66
- 18.9k stars
- pushed
- python-v4.2.4 on
- Apache-2.0
Install with pip
pip install -U deepevalWhen to use it, when to skip it
Why you'd want it
DeepEval works like Pytest for LLM apps: you write test cases and assert them against metrics such as G-Eval, task completion, tool correctness, answer relevancy, faithfulness and hallucination, then run them with
deepeval test run. Metrics cover agents, RAG, multi-turn chat, MCP use and multimodal output, can be powered by any LLM, and support both end-to-end and component-level evaluation. It also generates synthetic datasets, runs benchmarks like MMLU and GSM8K, and integrates with LangChain, LangGraph, CrewAI, OpenAI Agents, Pydantic AI and others.Best for
- Python teams that want evals in their test suite
- RAG and agent metrics
- CI checks on LLM output quality
Use it when
You want LLM quality checks written as tests that run locally or in any CI environment.
Skip it if
You are not working in Python (it requires Python 3.9+). Shared reports, iteration comparisons and production monitoring live in the separate Confident AI platform, which needs an account.
- Rank 58
Defines and runs model and agent evaluations in Python from datasets, solvers and scorers
- score 63
- 3k stars
- pushed
- 0.3.280 on
- MIT
Install with pip
pip install inspect-aiWhen to use it, when to skip it
Why you'd want it
Inspect, from the UK AI Security Institute and Meridian Labs, builds evaluations from composable datasets, solvers (from a single generate call to multi-turn agents with tools) and scorers, and runs them with
inspect eval. It supports custom and MCP tools plus built-in bash, python, web browsing and computer tools, can evaluate external agents like Claude Code, Codex CLI and Gemini CLI, and sandboxes untrusted model code in Docker, Kubernetes, Modal and other systems. It ships over 200 pre-built benchmark evals, support for over 20 model providers, the Inspect View log viewer and a VS Code extension.Best for
- Benchmarking models and agents
- Agentic evals that run code in sandboxes
- Research and safety evaluation teams
Use it when
You want to measure how a model or agent performs on a task suite, including agentic tasks with tools and sandboxed code, and compare results across providers.
Skip it if
You need tracing of a live production app; Inspect is an evaluation framework, and evals are written in Python.
- Rank 63
Traces, evaluates and monitors LLM apps and agents on LangChain's hosted platform via Python and JS SDKs
- score 60
- 1.1k stars
- pushed
- v0.14.8 on
- MIT
Install the Python SDK
pip install -U langsmith export LANGSMITH_TRACING=true export LANGSMITH_API_KEY=ls_...When to use it, when to skip it
Why you'd want it
LangSmith is LangChain's platform for debugging, evaluating and monitoring language model apps and agents; the MIT-licensed Python and JavaScript SDKs send traces to it, for example by wrapping the OpenAI client. It works with any LLM application and has a native integration with LangChain Python and LangChain JS. There is a free Developer plan with a monthly trace allowance and paid per-seat plans, with pay-as-you-go usage beyond the included traces.
Best for
- Teams already building with LangChain
- Hosted tracing without running infrastructure
- Python and JS/TS apps
Use it when
You want managed tracing and evals with no servers to run, especially if your app already uses LangChain.
Skip it if
You want to self-host without an enterprise contract; self-hosted and hybrid deployment are Enterprise plan options. Usage beyond the included traces is billed pay-as-you-go.