---
slug: "deepeval"
name: "DeepEval"
effect: "Unit tests LLM apps and agents in a Pytest-style workflow with LLM-as-a-judge and other metrics"
category: "eval-observability"
url: "https://deepeval.com"
github: "confident-ai/deepeval"
pricing: "open-source"
license: "Apache-2.0"
tags: ["evals", "python", "pytest", "testing", "rag", "agents"]
platforms: []
install_kind: "pip"
date_added: "2026-10-14"
drop_score: 36
stars: 18867
pushed_at: "2026-10-12T19:31:54Z"
signals_status: "ok"
signals_fetched_at: "2026-10-15T01:18:56.852Z"
page: "https://agenttooldrop.com/tools/deepeval"
---

# DeepEval

Unit tests LLM apps and agents in a Pytest-style workflow with LLM-as-a-judge and other metrics

Category: Evals and observability. Pricing: Open source. License: Apache-2.0.

## Why you would want it

DeepEval works like Pytest for LLM apps: you write test cases and assert them against metrics such as G-Eval, task completion, tool correctness, answer relevancy, faithfulness and hallucination, then run them with `deepeval test run`. Metrics cover agents, RAG, multi-turn chat, MCP use and multimodal output, can be powered by any LLM, and support both end-to-end and component-level evaluation. It also generates synthetic datasets, runs benchmarks like MMLU and GSM8K, and integrates with LangChain, LangGraph, CrewAI, OpenAI Agents, Pydantic AI and others.

## Best for

- Python teams that want evals in their test suite
- RAG and agent metrics
- CI checks on LLM output quality

## Use it when

You want LLM quality checks written as tests that run locally or in any CI environment.

## Skip it if

You are not working in Python (it requires Python 3.9+). Shared reports, iteration comparisons and production monitoring live in the separate Confident AI platform, which needs an account.

## Install

Install with pip:

```sh
pip install -U deepeval
```

Source: https://github.com/confident-ai/deepeval#readme

## Live signals

Fetched 2026-10-15T01:18:56.852Z (status: ok).

- Repository: confident-ai/deepeval
- Stars: 18,867
- Forks: 2,071
- Open issues and PRs: 728
- License (GitHub): Apache-2.0
- Last push: 2026-10-12
- Latest release: python-v4.2.4 (2026-09-22)

## Links

- Website: https://deepeval.com
- GitHub: https://github.com/confident-ai/deepeval
- Install instructions verified at: https://github.com/confident-ai/deepeval#readme
- Source for this entry: https://github.com/confident-ai/deepeval#readme
- AgentToolDrop page: https://agenttooldrop.com/tools/deepeval
- Full catalog (JSON): https://agenttooldrop.com/tools.json