Langfuse
ActiveDescription
Open-source LLM engineering platform providing tracing, evaluations, prompt management, and dataset management with integrations for LangChain, OpenAI, Anthropic, and more.
Key Features
- LLM application observability — trace LLM calls, retrieval, embeddings, agent actions with user session debugging
- Prompt management — centrally manage, version control, and iterate on prompts with server/client-side caching
- Evaluation pipelines — supports LLM-as-a-Judge, code evaluators, user feedback collection, and custom pipelines
- Datasets & benchmarks — create test sets and benchmark runs for continuous improvement and pre-deployment testing
- LLM Playground — jump directly from trace view to playground for prompt iteration and model config tuning
- ClickHouse-powered — built on ClickHouse for high-performance large-scale trace storage and querying
Use Cases
💡 Production LLM monitoring — real-time tracing of AI app call chains, latency, costs, and error rates
💡 Prompt A/B testing — manage multiple prompt versions, compare effectiveness through evaluation pipelines
💡 Model quality evaluation — automated scoring of model outputs using LLM-as-a-Judge
💡 Compliance audit & replay — full recording of all LLM interactions for post-hoc audit and issue debugging
💡 Cost optimization analysis — analyze token consumption and cost distribution across model calls
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (34.1k stars)
- • Established track record (3 years in production)
⚠️ Limitations
- • No clear open-source license
Categories
Quick Start
# Docker Compose local setup
git clone --depth=1 https://github.com/langfuse/langfuse.git
cd langfuse
docker compose up
# Python SDK integration
pip install langfuse
from langfuse import Langfuse
langfuse = Langfuse()
# Start tracing
trace = langfuse.trace(name='my-app')
span = trace.span(name='llm-call')
# ... execute LLM call ...
span.end()
langfuse.flush()