LLM Ops

LangDB

API Gateway

LangDB gives teams an AI gateway layer for model routing, caching, tracing, and policy control across LLM-powered applications.

Orq.ai

LLM Ops

LLM operations platform for managing prompts, models, and deployments in production with collaboration tools for AI product teams.

Martian

LLM Orchestration

Smart LLM router that automatically selects the optimal model for each prompt based on quality and cost.

Phospho

LLM Ops

LLM app analytics platform helping teams detect bugs and monitor user-LLM interactions in production.

Keywords AI

LLM Ops

Unified LLM API gateway and observability platform for monitoring, testing, and optimizing AI applications across multiple providers.

ThirdAI

AI Infrastructure

AI platform running LLMs on everyday CPUs with compression techniques for cost-effective inference.

Lunary

LLM Ops

Open-source LLM observability and analytics platform with logging, user analytics, and prompt templates.

Unify AI

LLM Ops

LLM routing platform benchmarking and routing requests to the best model-provider combination for each query.

Titan ML

AI Infrastructure

Enterprise-grade inference stack letting ML teams deploy, serve, and scale open-source LLMs in production.

Elyza

Generative AI Platforms

Japanese LLM company offering Japanese-language foundation models for enterprises.

Groq

AI/ML Infrastructure

Groq provides the fastest AI inference platform using custom LPU hardware, delivering ultra-low latency responses for LLM applications at competitive per-token pricing.

Okareo

AI Evaluation and Testing

AI application development platform for testing, evaluation, and deployment of LLM systems.

Log10

AI Observability

LLM observability and evaluation platform for monitoring, debugging, and improving AI application performance.

Traceloop

Observability Platforms

Open-source observability platform for LLM applications with tracing, monitoring, and prompt versioning via OpenLLMetry.

Guardrails AI

AI Guardrails

Open-source framework for adding validation and structure to LLM outputs with customizable quality checks.

Instructor

LLM Ops

Python library for getting structured outputs from LLMs using Pydantic models for reliable data extraction.

Velvet AI

API Gateway

AI gateway that logs, caches, and analyzes LLM requests for application developers.

Moonshot AI

AI Developer Tools

Chinese AI lab developing Kimi, a long-context LLM assistant with 2M token capacity.