Open sourcePython · ≈20k stars · Apache-2.0
DeepEval
The LLM evaluation framework — pytest for agents.
DeepEval grades LLM outputs with 20+ metrics (hallucination, faithfulness, task completion) inside pytest — unit-testing for prompts, RAG and agents, backed by Confident AI.
Activity
- Last push
- · 2d ago
- Latest release
- setup-v0.1.0
· 5w ago - Last 90 days
- 4 releases
setup-v0.1.0Setup wizard 0.1.0
· 5w agoWhat's Changed docs: Add missing params to Generate Goldens From Documents guide by @AbdoAlshoki2 in https://github.com/confident-ai/deepeval/pull/3311 . by @penguine-ip in https://github.com/confident-ai/deepeval/pull/3337 feat(persona): metadata and noise url by @tanayvaswani…
python-v4.2.4Introducing Jev in DeepEval
· 2mo agoNew to deepeval? Get started here. 🧠 Introducing Jev: System One for LLM Evals Every LLM-as-a-judge metric has had the same awkward secret: somewhere inside the thing measuring your LLM is another LLM generating text. Closed-ended verdicts (yes / no / borderline) were treated…
typescript-v0.9.13TypeScript 0.9.13Pre-release
· 2mo agoNew to deepeval? Get started here. ⚡ Improvement update changelogs for 2026 till August's latest release (#3106 by @A-Vamshi) 🐛 Bug Fix Refactor metrics to standardize score. (Higher is better) (#3102 by @A-Vamshi) A huge thank you to everyone who contributed to this release ❤️
python-v4.2.0Python 4.2.0
· 2mo agoNew to deepeval? Get started here. This release also collects the changes from 4.1.2 through 4.1.10, which shipped to PyPI without a GitHub release. ✨ New Feature Features/voice (#3036 by @penguine-ip) 4.1.10 support user credentials for Vertex AI (#3017 by @ykocaogullar) 4.1.8…
python-v4.1.9DeepEval - 4.1.9
· 3mo agoNew to deepeval? Get started here. ✨ New Feature add sync and async flushing of traces (#3045 by @tanayvaswani) type in tool-calls metrics (#3062 by @tanayvaswani) ⚡ Improvement Deepeval wizard (#3084 by @kritinv) Update setup-script.ts (#3085 by @kritinv) 🐛 Bug Fix Fix otel…
Checked against GitHub on Nov 21, 2026. Release notes are shortened to plain text.
Neighbours on the workbench
Go · ≈55k stars · Apache-2.0
alibaba/open-code-review
Alibaba's hybrid deterministic-plus-agent code review CLI.
Maintained by Alibaba
v1.13.13releasedPython · ≈39k stars · MIT
stanfordnlp/dspy
Programming, not prompting: optimize the whole pipeline.
Maintained by Stanford NLP
3.5.0releasedTypeScript · ≈38k stars · Other
langfuse/langfuse
Open-source LLM engineering platform: traces, evals, prompts.
Maintained by Langfuse
v4.83.0releasedConnections
Relations from the Harness Hub knowledge graph, as charted in the State of Harness reports.
Maintained by · 1
- Confident AIStartup
