Promptfoo
AI Evaluation and Testing
Open-source tool for testing and evaluating LLM prompts, models, and RAG pipelines with side-by-side comparisons.
Llm red teamingVulnerability scanningPrompt evaluation and testing
Alternatives to Ragas
- 1AI Evaluation and Testing
- 2AI Evaluation and Testing
Evidently AI
AI Evaluation and Testing
Open-source ML monitoring tool for evaluating, testing, and monitoring data and model quality in production.
Free planFree trialTiered plansData QualityData ObservabilityPrice$80 / month
Price position: below the category median. - 3AI Evaluation and Testing
BenchLLM
AI Evaluation and Testing
Open-source evaluation tool for testing and benchmarking LLM applications in CI/CD pipelines.
Free planContinuous TestingLLM Orchestration - 4AI Evaluation and Testing
AI Verify (IMDA)
AI Evaluation and Testing
Open-source AI governance testing framework by Singapore's IMDA for assessing AI system trustworthiness.
Free planAI GovernanceLLM Evaluation - 5AI Evaluation and Testing
Giskard
AI Evaluation and Testing
Open-source AI quality and testing platform for detecting vulnerabilities, biases, and hallucinations in ML models.
Automated adversarial red teaming for llm agentsLlm vulnerability detection and classificationDomain specific quality evaluation dataset generation - 6AI Evaluation and Testing
Confident AI
AI Evaluation and Testing
LLM evaluation platform providing automated testing and benchmarking for AI applications with custom metrics.
Llm evaluation with research backed metricsDataset management and auto curationGit based prompt versioning and management - 7AI Evaluation and Testing
Agenta
AI Evaluation and Testing
Open-source platform for prompt engineering, evaluation, and deployment of LLM applications with team collaboration.
Free planFree trialTiered plansAI AgentsAI Observability - 8AI Evaluation and Testing
Deepchecks
AI Evaluation and Testing
Open-source ML testing and monitoring platform for validating data and models throughout the development lifecycle.
Free planFree trialSOC 2GDPRHIPAATiered plans - 9AI Evaluation and Testing
Athina AI
AI Evaluation and Testing
LLM monitoring and evaluation platform detecting hallucinations, toxicity, and quality issues in production AI.
Free planSOC 2LLM OpsLLM Orchestration - 10AI Evaluation and Testing
Coxwave Align
AI Evaluation and Testing
LLM analytics platform providing automated evaluation, monitoring, and insights for AI products.
User behavior analysis for AI productsQuantitative evaluation metrics for AI products
