LLM Evaluation

Braintrust

AI Evaluation and Testing

Braintrust is an AI evaluation platform for testing prompts, models, and application behavior with production-like datasets and scoring workflows.

LangSmith

AI Observability

LangSmith is LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications and agent workflows.

Scorecard

AI Evaluation and Testing

LLM testing and evaluation platform with continuous testing pipelines.

Confident AI

AI Evaluation and Testing

LLM evaluation platform providing automated testing and benchmarking for AI applications with custom metrics.

BenchLLM

AI Evaluation and Testing

Open-source evaluation tool for testing and benchmarking LLM applications in CI/CD pipelines.

Agenta

AI Evaluation and Testing

Open-source platform for prompt engineering, evaluation, and deployment of LLM applications with team collaboration.

Mercor

Talent Marketplace

AI recruiting platform that uses video interviews to match candidates with tech companies.

Parea

AI Observability

LLM observability and evaluation platform for AI teams with prompt testing and production monitoring.

Promptfoo

AI Evaluation and Testing

Open-source tool for testing and evaluating LLM prompts, models, and RAG pipelines with side-by-side comparisons.

LastMile AI

AI Developer Tools

Developer platform for building, evaluating, and deploying generative AI applications to production.

Trulens

LLM Ops

Open-source library for evaluating and tracking LLM and RAG application quality with feedback functions.

Ragas

AI Evaluation and Testing

Open-source framework for evaluating retrieval-augmented generation pipelines with automated quality metrics.

Okareo

AI Evaluation and Testing

AI application development platform for testing, evaluation, and deployment of LLM systems.

RagaAI

AI Evaluation and Testing

AI testing platform for debugging and evaluating ML and generative AI systems.

Log10

AI Observability

LLM observability and evaluation platform for monitoring, debugging, and improving AI application performance.