Dify

Dify is an open-source LLM app development platform. It provides a visual interface for building AI workflows, RAG pipelines, and agentic applications, allowing users to move from prototype to production within a single environment. The platform simplifies the orchestration of large language models by combining model management, prompt engineering, and observability into one dashboard.

The software is deployed as a server-side application, typically using Docker Compose for local or private server installations. It is also available as a cloud service for users who prefer a managed environment without manual setup. For those requiring high availability, the platform supports deployment via Kubernetes, Helm Charts, and various cloud infrastructure tools including Terraform and AWS CDK.

Key features

• Visual canvas for building and testing AI workflows
• Integration with hundreds of proprietary and open-source LLMs
• Prompt IDE for crafting prompts and comparing model performance
• RAG pipeline with text extraction for PDFs and PPTs
• Agent capabilities based on LLM Function Calling or ReAct
• Over 50 built-in tools including Google Search and WolframAlpha
• LLMOps for monitoring application logs and performance
• Backend-as-a-Service APIs for integration into external business logic

Dify is designed for developers and organizations building generative AI applications. It supports a wide range of inference providers and any OpenAI API-compatible models, including GPT, Mistral, and Llama3. For observability and performance tracking, it integrates with tools such as Opik, Langfuse, and Arize Phoenix. Users can monitor metrics at the level of apps, tenants, and messages by importing a dedicated dashboard into Grafana using a PostgreSQL data source.

The platform is built to handle the full lifecycle of AI development, from initial prompt experimentation in the IDE to the deployment of complex agentic workflows. It provides the necessary infrastructure to manage datasets and refine models based on production data and annotations.

It is a comprehensive orchestration layer that combines model management and workflow automation for AI agents.