Memovable：記憶體內 LLM 上下文最佳化引擎

Memovable — In-Memory LLM Context Optimization Engine

Elevator Pitch

An open-source, in-memory context optimization engine that prunes bloated LLM context windows in real-time—slashing token usage, reducing API costs, and speeding up response times for developers and businesses.

Inspiration

LLM API costs scale directly with token usage. For startups, freelancers, and businesses running automated agents or customer service bots, passing thousands of tokens of accumulated conversation context on every single query quickly becomes financially unsustainable. We built Memovable to solve this exact problem: selectively stripping out irrelevant background context while keeping the core intent and system instructions 100% intact before the prompt ever hits the model.

What it does

Memovable acts as an intelligent, high-speed middleware layer between the user application and the LLM API (currently integrated with Google Gemini):

1. Input Normalization: Takes the raw userQuery alongside the existingLLMContext.
2. Intent & Keyword Extraction: Normalizes the incoming query, identifies user intent, and extracts key semantic markers.
3. Smart Context Pruning: Cross-references extracted keywords and intent against the existing context stack to strip out unneeded tokens, repetitive fluff, and stale information.
4. Optimized Model Execution: Passes only the pruned, high-relevance context and original query to the LLM, delivering identical output quality at a fraction of the token cost and latency.

How we built it

• Core Engine: Built an in-memory optimization pipeline in TypeScript/Node.js to handle normalization, string tokenization, and semantic relevance matching with zero database overhead.
• LLM Integration: Integrated with the Google Gemini API to benchmark real-time token reduction and response latency improvements.
• Documentation & Web Hub: Published full documentation and setup guides live at https://memovable.tech/.

Why Open Innovation Matters for Memovable

Closed, proprietary API ecosystems force developers into expensive, black-box billing where every extra token costs real money. Open-source innovation makes Memovable fundamentally better than closed alternatives:

• Zero Lock-In & Total Privacy: Because the context pruning logic is completely open-source, developers and small businesses can run Memovable locally on their own infrastructure or laptops. Sensitive customer data and internal logs never leave the owner's environment just to be optimized.
• Business Viability (The Side Hustle Edge): Running an AI-based SaaS or customer agent often fails because API costs eat into thin profit margins. Memovable keeps LLM token consumption low enough that small AI side hustles and freelance developer tools become immediately profitable.
• Model Flexibility: Being open-source allows developers to swap underlying models seamlessly—from cloud endpoints like Gemini to local open-weight models like Gemma—without rewriting their context-pruning architecture.

Challenges we ran into

At the event, we had to pivot our original plan at the very last minute to comply with hackathon rules requiring a build completely from scratch. Designing, coding, benchmarking, and publishing the full engine alongside documentation at memovable.tech within a strict 3-hour sprint required aggressive scoping and laser focus on core execution.

Accomplishments that we're proud of

• Built a functional, end-to-end working prototype from scratch under intense time constraints.
• Successfully demonstrated a real-time token reduction benchmark on live Gemini API queries.
• Deployed documentation live at https://memovable.tech/.
• Secured the top hardware award (Arduino UNO Q) and successfully pitched the project during a technical interview session with event hosts.

What's next for Memovable

• Adding support for local open-weight models (Gemma via Ollama) for 100% offline context pruning.
• Building custom NPM and PyPI client libraries for 1-line middleware integration.
• Implementing dynamic semantic embeddings for higher-accuracy context filtering in multi-turn agent workflows.