RWKV Runner

RWKV Runner is an open-source desktop app that provides a management and startup tool for RWKV large language models. It automates the deployment process through a lightweight executable, allowing users to run models locally without the need for manual dependency installation. The software is available for Windows, macOS, and Linux, making it accessible across the most common desktop platforms.

The application functions as both a model runner and an API client. It provides a dedicated user interface for chatting and text completion while simultaneously hosting a backend that is compatible with the OpenAI API. This compatibility allows existing clients to connect to the local RWKV instance. Users can also configure the software to act as a client for external services by entering the relevant API URL and key in the settings page.

Key features

• One-click startup and management for RWKV models
• OpenAI API compatible interface for integration with other clients
• Automatic dependency installation via a small executable
• Support for WebGPU to run on AMD and Intel graphics cards
• Built-in model conversion and download management tools
• One-click LoRA Finetuning for Windows users
• Integrated chat, completion, and composition interfaces
• MIDI hardware input support for musical composition
• Front-end and back-end separation for server deployment

The software is designed for users who want to run open-source LLMs locally with minimal configuration. It includes pre-set VRAM strategies to ensure compatibility across various hardware specifications and offers a custom CUDA kernel for faster inference and lower memory usage. The architecture supports a standalone desktop experience or a split deployment where the backend runs on a remote server while the desktop app serves as the client. Additionally, it includes a built-in WebUI option to share hardware resources as a web service.

RWKV-Runner positions itself as a bridge between complex machine learning infrastructure and end-user accessibility.