Sulci Labs 示範如何把 Sulci 語義快取接入 AI 智能體，從註冊到儀錶板

An AI agent asks the model the same things every time it runs the same task: the same plan step, the same "did that work?" check, the same summary of the same docs. We recorded the whole path of putting the Sulci semantic cache in front of one, from signup to dashboard, in six minutes:

This post covers the parts worth reading slowly: the three code changes, the agent settings, and the moment the cache chose not to answer.

The agent

The demo is a planner–reflector loop: plan a step, act, then ask the model "did that step succeed?" Its task is "deploy the web service to AWS", and every plan step and every check is a call to Claude. Run the same task tomorrow and it makes the same calls again.

Each prompt carries its own state: the task, the steps already done, and the last tool output.

You are an operations agent. Task: deploy the web service to AWS.
Completed steps: configure AWS credentials. Reply with the single next step to perform.

Three code changes

import anthropic
import sulci
from sulci import Cache

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY
MODEL = "claude-haiku-5-5"              # any Claude model works

cache = Cache(backend="sulci")          # 1. reads SULCI_API_KEY from the environment

def ask_claude(prompt):
    msg = client.messages.create(model=MODEL, max_tokens=512,
                                 messages=[{"role": "user", "content": prompt}])
    return sulci.with_usage(msg.content[0].text, msg.usage)   # 3. token counts for the dashboard

def plan_step(prompt):
    result = cache.cached_call(prompt, ask_claude,           # 2. route the call through the cache
                               standalone=True, threshold=0.95)
    return result["response"]

1. Create a cache. backend="sulci" is the managed tier; the free tier needs no card. Self-hosted backends (SQLite and others) work the same way; Tokens Saved on the dashboard is a managed-tier feature.
2. Route the call through cached_call. On a hit you get the stored answer. On a miss Claude answers and the cache keeps the answer.
3. Return the answer with with_usage. It attaches the call's token counts, so the dashboard can show the tokens each hit saved.

Agent-safe settings

• standalone=True: the prompt carries everything needed to answer it, so its answer is stored even when other calls came before it.
• threshold=0.95: strict. Agent prompts often differ in one detail (step 2 versus step 3, exit code 0 versus 1), and a near-miss must go to the model, not get another step's answer.
• No session_id. With a session, the lookup blends the query with the conversation's recent turns, which is what lets a vague follow-up like "how do I fix it?" search with its conversation's context. Agent prompts already carry their own state, so they're looked up on their own: earlier steps can't pull a prompt toward another step's answer.

The moment that matters: a cache that knows when not to answer

In run 3, one step's tool output is a failure:

You are an operations agent. Task: deploy the web service to AWS. Step just performed: configure AWS
credentials. Tool output: exit code 1; AccessDenied: not authorized to perform this action; request id
req-b41e5c. Did the step succeed? Answer yes or no with a reason.

A naive cache would hand back run 1's "Yes, it succeeded." Here the failed prompt scored below the threshold against the cached success, so it missed, and Claude answered "No, the step did not succeed…". In our offline replay of the same prompts with the same embedder, it scores 0.927 against the cached success: under the 0.95 bar. At the hosted default of 0.65 it would have been served, which is why agent prompts get a strict threshold. Catching repeats is the easy part; knowing when not to answer is the hard part.

The numbers (one run, not a benchmark)

• Run 1: every step went to Claude.
• Runs 2 and 3: mostly served from the cache; repeats at similarity 1.000, reworded checks at 0.985–0.991.
• Overall: 11 of 18 steps came from the cache, and the failure step went to Claude.
• The dashboard: requests, hit rate, Tokens Saved (the tokens of the avoided calls, as the app reported them), and Cost Saved. Cost Saved is an estimate: hits × the cost per call you set.

That is one run of a demo app. What you save depends on how repetitive your traffic is.

Try it

pip install sulci anthropic

• Free tier, no card: sulci.io
• Source (Apache 2.0): github.com/sulci-io/sulci-oss
• More demos: youtube.com/@sulci_cache