Know when your AI is wrong

Braintrust

AI Evaluation and Testing

Braintrust is an AI evaluation platform for testing prompts, models, and application behavior with production-like datasets and scoring workflows.

LangSmith

AI Observability

LangSmith is LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications and agent workflows.

Langfuse

AI Observability

Langfuse is an open-source platform for tracing, prompt management, evaluations, and observability across LLM applications and agent systems.

Arize

Observability Platforms

Arize helps teams monitor, evaluate, and improve LLM applications, retrieval systems, and machine learning products once they are live in production.

Mosaic AI

AI Infrastructure

Databricks' Mosaic AI suite for building, tuning, and serving LLMs with RAG, evaluation, and monitoring.

PromptLayer

Prompt Engineering Tools

PromptLayer is a platform for prompt management, AI observability, evaluation workflows, and release control across LLM apps and agent systems.

Maxim AI

AI Evaluation and Testing

End-to-end AI evaluation and monitoring platform for LLM product teams with prompt management and datasets.

Promptfoo

AI Evaluation and Testing

Open-source tool for testing and evaluating LLM prompts, models, and RAG pipelines with side-by-side comparisons.

Traceloop

Observability Platforms

Open-source observability platform for LLM applications with tracing, monitoring, and prompt versioning via OpenLLMetry.

LangWatch

AI Observability

LangWatch helps teams monitor prompts, traces, and quality signals across production LLM applications and agent workflows.

Opik

AI Observability

Open-source LLM evaluation and observability platform from Comet.

LLMonitor

AIOps Platforms

Open source observability platform for AI apps with prompt tracking

Parea

AI Observability

LLM observability and evaluation platform for AI teams with prompt testing and production monitoring.

Confident AI

AI Evaluation and Testing

LLM evaluation platform providing automated testing and benchmarking for AI applications with custom metrics.

LLM observability and evaluation platform for building production-grade AI applications with tracing, monitoring, and annotation tools.

Log10

AI Observability

LLM observability and evaluation platform for monitoring, debugging, and improving AI application performance.

Fiddler AI

AI Observability

AI observability platform for monitoring, explaining, and analyzing ML model predictions in production.

Helicone

Monitoring & Observability

Helicone is an open-source LLM observability platform providing request logging, cost tracking, caching, and evaluation tools for AI applications in production.

Evidently AI

AI Evaluation and Testing

Open-source ML monitoring tool for evaluating, testing, and monitoring data and model quality in production.

Trubrics

AI Evaluation and Testing

LLM observability platform for collecting user feedback, evaluating model performance, and analytics for GenAI apps.

Orq.ai

LLM Ops

LLM operations platform for managing prompts, models, and deployments in production with collaboration tools for AI product teams.

PromptArmor

AI Security Platforms

AI security platform defending LLM applications from prompt injection and exploits.

LangTail

Prompt Engineering Tools

LLM testing and deployment platform with prompt playground and observability for AI developers.

Vellum

Prompt Engineering Tools

Vellum helps teams design, test, and orchestrate prompt-driven workflows and LLM-powered products with a more controlled production pipeline.

Lunary

LLM Ops

Open-source LLM observability and analytics platform with logging, user analytics, and prompt templates.

UpTrain

AI Evaluation and Testing

Open-source LLM evaluation and observability tool for production AI pipelines.

Klu

LLM Ops

LLM app platform with prompt engineering, evaluation, and observability for teams

Galileo Observe

AI Observability

ML observability platform for monitoring models in production with drift and performance tracking.

Laminar

AI Observability

Open source observability and evaluation platform for LLM applications with tracing.

Flytrap

AI Infrastructure

AI observability platform for monitoring errors and debugging issues in LLM applications.