BenchLLM
AI Evaluation and Testing
Open-source evaluation tool for testing and benchmarking LLM applications in CI/CD pipelines.
Future of SoftwareStart Survey
AI Evaluation and Testing
Open-source evaluation tool for testing and benchmarking LLM applications in CI/CD pipelines.

LLM Orchestration
Flowise is an open-source UI visual tool for building customized LLM orchestration flows and AI agents using a drag-and-drop interface with LangChain integration.
Price$35+ / month

AI Observability
Open source observability and evaluation platform for LLM applications with tracing.

LLM Orchestration
Python toolkit for building LLM applications with clean abstractions for prompts, calls, and structured extraction.

LLM Orchestration
Pipecat is an open-source framework for building real-time voice and multimodal AI agents with transport, pipeline, and interaction primitives.

Conversational AI
Open-source framework for building conversational AI interfaces with LLM applications and agent workflows.

LLM Orchestration
LLM orchestration library for building production AI applications with Python decorators.
LLM Orchestration
LLM observability and evaluation platform for building production-grade AI applications with tracing, monitoring, and annotation tools.

AI Agents
Mastra is a TypeScript framework for building AI agents, workflows, and retrieval systems with a developer-first, production-oriented approach.
LLM Orchestration
Open-source embeddings database for semantic search, LLM orchestration, and language model workflows.