Helicone

Helicone is an open-source and self-hosted web app and server that provides observability for large language model applications. It functions as an AI gateway and monitoring platform, allowing developers to log, debug, and analyze requests sent to various AI models. The software helps engineers manage the lifecycle of AI applications by providing visibility into how models behave in production environments.

The software is deployed as a server application via Docker or Helm charts for enterprise workloads. It integrates into existing workflows by updating the baseURL in the application code, enabling the capture of requests from providers such as OpenAI, Anthropic, and Gemini, as well as frameworks like LlamaIndex. This approach allows for the collection of telemetry data without requiring extensive changes to the underlying application logic.

Key features

• AI Gateway providing a unified API for over 100 models with automatic fallbacks
• Request logging for OpenAI, Anthropic, Gemini, and Vercel AI SDK
• Trace and session inspection for agents and chatbots
• Tracking for cost, latency, and quality metrics
• UI playground for testing prompts and sessions
• Prompt management with versioning and deployment without code changes
• Data export to PostHog for custom dashboards
• Integration with fine-tuning partners OpenPipe and Autonomi

The architecture consists of five primary services to handle high volumes of telemetry data. It uses a NextJS frontend for the platform interface, Cloudflare Workers for proxy logging, and an Express server for log collection. Data persistence is managed through Supabase for application database and authentication, ClickHouse for high-performance analytics, and Minio for object storage of logs. This stack is designed for AI engineers who need to monitor document processing pipelines, chatbots, and complex agentic workflows.

Helicone is an LLMOps tool focused on observability and prompt engineering for production AI applications.