Dagu

Dagu is an open-source, self-hosted workflow engine that provides a web interface for defining and monitoring declarative YAML based data pipelines. It functions as a lightweight orchestrator for scheduling tasks, managing dependencies, and ensuring durable execution of automation scripts across various environments.

The software is deployed as a single binary and supports Linux, macOS, and Windows. It can be run as a local single-server instance for development, as a self-hosted server with distributed workers for larger workloads, or via Docker and Kubernetes. Users define Directed Acyclic Graphs (DAGs) in YAML to organize shell commands, containers, and remote scripts into observable pipelines without needing to modify their existing tools.

Key features

• Declarative YAML format for defining workflow DAGs
• Web UI for visual management, monitoring, and retrying pipelines
• Support for shell commands, Docker containers, and Kubernetes Jobs
• Remote command execution via SSH
• Built-in MCP server for AI agent integration
• Human-in-the-loop steps for manual approvals and interventions
• Local file-based state storage requiring no external database
• Integration with GitHub events to trigger workflows

Dagu is designed for small teams and DevOps engineers who need to add scheduling and observability to existing scripts without writing new code. It integrates with external coding-agent CLIs through a built-in harness and supports various execution models, including private workers over mTLS for hybrid deployments. The architecture allows for load control through queues, concurrency limits, and resource constraints, enabling the system to handle thousands of workflow runs per day depending on the hardware.

Typical use cases include ETL and data operations, media conversion using ffmpeg, and IoT edge workflows where jobs must remain close to the data source. It also enables the creation of self-service support portals, allowing non-engineering teams to run approved diagnostic workflows and database queries without direct engineering intervention.

It serves as a tool for ETL operations, infrastructure automation, and self-service support portals.