Harbor

Harbor is an open-source application that provides a CLI and companion app for deploying and orchestrating a local LLM stack. It manages the deployment of backends, frontends, and supporting AI services using Docker Compose, ensuring that different components are pre-wired for connectivity without manual configuration. This removes the need for hand-editing provider configurations when setting up complex local environments.

The software is used to spin up various inference engines, web interfaces, and agentic tools on a local server or homelab. It is deployed as a CLI tool that interacts with Docker to manage containers and can also be used via a companion application for guided installation and service management. Users can launch specific services or entire stacks with single commands to establish a functional AI workspace.

Key features

• Orchestrates backends including Ollama, llama.cpp, and vLLM
• Deploys frontends such as Open WebUI and ComfyUI
• Integrates SearXNG for local web RAG and deep research
• Supports host-native Metal and GPU inference on macOS via MLX and DMR
• Provides Harbor Boost for agentic coding workflows and research presets
• Connects local backends to coding tools like Codex and VS Code via harbor launch
• Manages MCP ecosystem tools through MetaMCP
• Includes a built-in tunneling service to expose local instances to the internet
• Generates QR codes for quick mobile access to local services

Harbor is designed for developers and AI enthusiasts who want to run a complete generative AI stack locally. It simplifies the complexity of cross-service connectivity by handling the networking and environment variables required for tools like Speaches for voice chat or Dify for LLM workflows. The architecture supports both containerized services and host-native compute for optimized performance on specific hardware, such as Apple Silicon. Additionally, the Harbor Boost module allows for the chaining of agentic modules for tasks like web research, deliverable audits, and scoped file edits, which can be mounted directly to project repositories.

It serves as a management layer for self-hosted AI infrastructure, focusing on the automation of deployment and configuration for local model inference.