以 Rust 建構 CedroDB：探索供 AI 智能體使用的本地優先儲存

Building CedroDB: Exploring Local-First Storage for AI Agents with Rust 🚀

Modern AI applications are becoming increasingly complex.

An AI agent may need to remember previous interactions, retrieve relevant information, maintain its execution state, and recover interrupted workflows.

These capabilities often require combining databases, search engines, job queues, and persistence mechanisms.

But what if we could explore bringing several of these capabilities into a single embedded storage engine?

That's the idea behind CedroDB.

🌱 What Is CedroDB?

CedroDB is an experimental, open-source, local-first database engine developed in Rust, with native libraries for Python and Node.js/TypeScript.

It explores combining document storage, search, durable workflows, and persistent agent state without requiring a separate database server.

The goal isn't to replace established databases. It's to experiment with a storage architecture designed around the needs of local applications and AI agents.

🧠 The Challenge: Persistent State for AI Agents

Imagine building an AI assistant that needs to:

• Remember information between sessions.
• Search stored documents and knowledge.
• Save execution checkpoints.
• Resume interrupted operations.
• Process background tasks reliably.

Implementing these capabilities usually involves coordinating multiple components.

CedroDB explores an alternative: managing them through a unified local storage engine.

One particularly interesting capability is storing an agent checkpoint together with related document and outbox writes in a single atomic transaction.

This can help avoid inconsistent local state when a workflow is interrupted.

⚙️ What's Inside CedroDB?

1. JSON Document Storage

Store structured JSON documents and perform atomic operations across collections.

2. Full-Text and Vector Search

CedroDB supports:

• BM25-based full-text search.
• Exact vector similarity search.
• Hybrid retrieval combining text and vector search.

This allows applications to retrieve local information using different search strategies.

The application remains responsible for generating embeddings.

3. Persistent Agent Checkpoints

Agent checkpoints allow applications to preserve execution state.

CedroDB also supports revision checks to prevent stale checkpoint updates.

4. Durable Background Jobs

Persistent job queues support retries, leases, and recoverable workflows.

Applications remain responsible for running workers and handling external operations.

5. Changefeeds and Backups

CedroDB includes durable changefeeds, backup capabilities, and integrity verification mechanisms.

These features help support local integrations and recovery workflows.

🦀 Why Rust?

Rust provides several characteristics that make it interesting for building storage engines:

• Memory safety without garbage collection.
• Explicit ownership and resource management.
• Native performance.
• Strong support for systems programming.
• Cross-platform development.

CedroDB uses Rust for its core engine while exposing native bindings for other programming languages.

This allows Python and JavaScript developers to experiment with the same underlying storage system.

🐍 Using CedroDB with Python

Install the package:

pip install cedrodb

Example:

from cedrodb import Database

with Database("./app.cedro") as db:
    notes = db.collection("notes")

    notes.put_many({
        "note-1": {"state": "pending"},
        "note-2": {"state": "completed"}
    })

    notes.create_index("state")

    results = notes.find_page([
        {
            "field": "state",
            "op": "eq",
            "value": "pending"
        }
    ])

    print(results["records"])

No external database server is required.

📦 Using CedroDB with Node.js

Install CedroDB from npm:

npm install cedrodb

The native Node.js/TypeScript bindings provide access to the Rust storage engine.

You can explore the API and examples directly in the GitHub repository.

🔬 Experimental by Design

CedroDB is currently an experimental project (v0.9.0).

There are important limitations:

• Active documents and indexes are stored in RAM.
• One process owns a database directory at a time.
• Vector retrieval currently uses exact search rather than ANN indexes.
• SQL and MVCC are not supported.
• Replication is not available.
• Production readiness has not been established.

These trade-offs are part of the project's ongoing exploration.

SQLite and other mature databases remain better choices for many production workloads.

🚀 What's Next?

The roadmap toward CedroDB 1.0 focuses on validation, reliability, performance testing, and improvements to the storage engine.

Community feedback is particularly valuable in identifying architectural limitations and practical use cases.

🔗 Explore the Project

💬 Join the Discussion

Building an embedded database is an interesting engineering challenge, especially when combining storage, retrieval, and recoverable workflows.

I'd love to hear your thoughts:

Would you use a single embedded storage engine for AI agent memory, document search, and durable jobs, or would you prefer separate specialized components?

Feedback, technical discussions, and open-source contributions are welcome.

If you find CedroDB interesting, consider giving the repository a ⭐ on GitHub.

Built with Rust. Accessible from Python and Node.js. Open source.

rust #opensource #database #ai