BenchLLM
AI Evaluation and Testing
Open-source evaluation tool for testing and benchmarking LLM applications in CI/CD pipelines.
Free planContinuous TestingLLM Orchestration
Alternatives to Promptfoo
- 1AI Evaluation and Testing
- 2AI Evaluation and Testing
Giskard
AI Evaluation and Testing
Open-source AI quality and testing platform for detecting vulnerabilities, biases, and hallucinations in ML models.
Automated adversarial red teaming for llm agentsLlm vulnerability detection and classificationDomain specific quality evaluation dataset generation - 3AI Evaluation and Testing
Evidently AI
AI Evaluation and Testing
Open-source ML monitoring tool for evaluating, testing, and monitoring data and model quality in production.
Free planFree trialTiered plansData QualityData ObservabilityPrice$80 / month
Price position: below the category median. - 4AI Evaluation and Testing
Ragas
AI Evaluation and Testing
Open-source framework for evaluating retrieval-augmented generation pipelines with automated quality metrics.
Automatic llm driven metrics for rag evaluationCustom metric creation and adaptationFramework and tool integrations - 5AI Evaluation and Testing, Prompt Engineering Tools
Agenta
AI Evaluation and Testing
Open-source platform for prompt engineering, evaluation, and deployment of LLM applications with team collaboration.
Free planFree trialTiered plansAI AgentsAI Observability - 6AI Evaluation and Testing
Deepchecks
AI Evaluation and Testing
Open-source ML testing and monitoring platform for validating data and models throughout the development lifecycle.
Free planFree trialSOC 2GDPRHIPAATiered plans - 7AI Evaluation and Testing
Braintrust
AI Evaluation and Testing
Braintrust is an AI evaluation platform for testing prompts, models, and application behavior with production-like datasets and scoring workflows.
Real time trace inspection and loggingLlm based and custom code evaluationExperiment management and comparisonPrice$249 / month
Price position: above the category median. - 8AI Evaluation and Testing
AI Verify (IMDA)
AI Evaluation and Testing
Open-source AI governance testing framework by Singapore's IMDA for assessing AI system trustworthiness.
Free planAI GovernanceLLM Evaluation - 9AI Evaluation and Testing
Okareo
AI Evaluation and Testing
AI application development platform for testing, evaluation, and deployment of LLM systems.
Synthetic user simulation with multi turn conversationsVoice and text agent evaluationProduction monitoring and closed loop feedback - 10AI Evaluation and Testing
Confident AI
AI Evaluation and Testing
LLM evaluation platform providing automated testing and benchmarking for AI applications with custom metrics.
Llm evaluation with research backed metricsDataset management and auto curationGit based prompt versioning and management
