Mintu：使用 Tauri 2、Rust 和 TypeScript 打造開源桌面寵物與本地 AI 助手

Most desktop AI interfaces today follow a predictable pattern: an isolated browser tab, a heavy Electron window, or a generic sidebar that gets lost in a sea of open windows.

With Mintu, I wanted to explore a different paradigm: what if your AI assistant was not an application you open and close, but an ambient, living desktop companion that resides naturally in your workflow?

Mintu is an open-source living desktop pet and local AI assistant built from scratch using Tauri 2, Rust, Vite, and TypeScript. It combines the ergonomics of an Apple-style Dynamic Island with 60 FPS kinematic pet physics, 100% private local LLM execution, real-time screen intelligence, and an open plugin ecosystem.

Architecture Overview

Building an interactive desktop pet that overlays other full-screen applications without stealing input focus or causing high CPU consumption requires strict architectural separation:

Unified Click-Through Screen Overlay Window (Tauri 2 / Win32):
Single transparent coordinate space across the primary monitor with a Dynamic Island using spring-physics morphing (width, height, radius), a kinematic desktop pet layer (gravity, momentum, window edge anchoring), and a 30 Hz Win32 cursor poller with dynamic DOM hit-rect synchronization (maintaining under 2% idle CPU).
Local AI Engine (llama.cpp Sidecar):
Automatic hardware tier detection (RAM, VRAM, CPU cores), a resumable chunk downloader with SHA-256 hash validation, real-time token streaming with syntax-highlighted markdown, and multi-provider cloud adapters (Gemini, Groq, OpenAI, Ollama).
Ambient Intelligence & Smart Clipboard:
Foreground window process inspection and security blocklist, interactive screen-region capture with offline Windows Media OCR, and a real-time clipboard classifier (code review, translation, JSON cleanup).
Extensibility & Gamification:
Focus XP progression and customizable wardrobe accessories, paired with JSON-based skin templates and runtime plugin manifests.

Key Features

1. Dynamic Island & Kinematic Pet Physics

Mintu lives inside a floating pill at the top of your screen that expands, contracts, and morphs into different states (notifications, system media, Pomodoro timer, AI chat).

When you want Mintu to explore, you can drag Mintu out or press Ctrl + Alt + M. Mintu leaves the island, sits on top of your currently active window, falls down if that window closes, and hops back into the island when thrown near the magnetic docking zone.

2. 100% Local & Privacy-First AI

Privacy is a core design principle:

Integrated llama.cpp sidecar running quantized GGUF models directly on your CPU/GPU.
No network requests are sent when using local models.
Built-in Privacy Guardian automatically suppresses context ingestion when sensitive applications (password managers, banking tools) are in the foreground.

3. Smart Clipboard

Whenever you copy text with Ctrl + C, Mintu analyzes the content in real time and offers context-sensitive actions directly in the island:

Programming code: Suggests "Check code for errors" or "Explain code".
Foreign-language text: Suggests "Translate to Azerbaijani" (or your preferred language).
JSON or unstructured data: Suggests "Format and validate JSON".

4. Multi-Language Voice & Vision

Microphone speech-to-text with automatic multi-language recognition (supporting Azerbaijani, English, Russian, Turkish, and more).
Interactive screen-region capture (Ctrl + Alt + S) that captures any rectangle on your screen and extracts text using Windows Media OCR without uploading images to any third-party server.

5. Focus Gamification & Wardrobe

As you complete Pomodoro work intervals, Mintu earns focus XP and levels up. You can customize Mintu with earned accessories from the wardrobe, including reading glasses, winter beanies, a coffee mug, and noise-cancelling headphones.

6. Developer Extensibility: Plugins & Custom Skins

Mintu v0.2.0 introduces an open architecture:

Plugins: Developers can build standalone widgets (crypto/currency rates, GitHub alerts, prayer times, weather).
Custom Skins: A transparent JSON schema allowing artists and developers to reskin Mintu with their own spritesheets and animations.

Getting Started

You can test the pre-compiled standalone binary or build from source:

Prerequisites

Windows 10 (version 1809+) or Windows 11
Microsoft Edge WebView2 Runtime (pre-installed on modern Windows)
Node.js 18+ and Rust toolchain (if building from source)

Installation

Clone the repository:


bash
git clone https://github.com/HuseynDeveloper/Mintu.git
cd Mintu
npm install