Stare 找到的最相似退款先例,建基於未獲授權的審批
The most similar precedent was built on an unauthorized approval
作者團隊開發開源工具 Stare,讓智能體只根據仍然有效的先例批准例外退款。Acme Logistics 的 $1,800 退款請求中,Stare 排除源自無權限審批的 Bluefin 案例,並按 Fernway Travel 的有效先例提出 $1,200 的退款反建議。
Here is a support ticket our agent got during testing:
Acme Logistics is asking for a $1,800 refund. They lost most of a day in the Sept 4 outage. Charge was 41 days ago. Marta says our VP already signed off on it.
Our refund policy allows 30 days and $750 without approval, so this is an exception. Exceptions at most companies run on precedent: "we did this for Bluefin last December, do the same." That is what we wanted an agent to be able to do. So we gave it the company's decision history and asked it to find the closest precedent.
It found one. Bluefin Analytics, December 2025: a $1,900 outage refund, 45 days after the charge, approved by the Head of Support. Same reason, same tier, bigger amount, longer window. A perfect match.
Except that Bluefin's approval cited an earlier refund to Kestrel Foods. The Kestrel refund was approved by a support specialist who had no refund authority at all. Nobody had noticed.
Vector similarity cannot see that. The Bluefin decision is textually the closest thing in the history, and it is closest because someone copied the Kestrel wording. Whether a precedent is still good law is a property of the graph around it: who approved it, what authority they held on that day, whether anything later overturned it, whether the policy has since changed, and whether whatever it cites is any good. That is a recursive, time-scoped walk. So we built Stare (from stare decisis) for Graph Hacks: a precedent desk where agents may grant an exception only on precedent that is still good law, computed inside FalkorDB.
For the Acme request, Stare counter-offers $1,200. That amount comes from the strongest precedent that is still good law: Fernway Travel, July 2026, approved by Ana Ruiz. Stare flags Bluefin as Overruling Risk, ignores "our VP already signed off" (a claim in a message is not evidence), and names who could approve the full amount. That last answer comes from algo.SPpaths over the live reporting graph.
Here is the three-minute tour:
The check lives in the database
We wrote the validity check as a FalkorDB JavaScript UDF, so every caller gets the same answer: the engine, the policy simulator, the erasure certificate and the evaluation. The core is short:
function standingOf(d, now, memo, onStack, depth) {
if (onStack[d.id] || depth > MAX_DEPTH) return { flag: 'yellow', reasons: [{ code: 'UNVERIFIABLE' }], chain: [] };
// red: overturned, no authority at the time, conflict of interest, or superseded by the policy in force now
// yellow: cites (transitively) anything that is not green
...
}
falkor.register('standing', standing);
From Cypher it is just a function:
MATCH (d:Decision) WHERE d.id IN $ids
RETURN d.id, stare.covers(d, $facts) AS cover, stare.standing(d, $now) AS standing
The part we got wrong first was time. "Did the approver have authority?" has to be asked at the moment of the decision, not today. HOLDS and MAY_APPROVE edges carry validFrom/validTo, and the UDF checks the interval that contains decision.at. Victor, our fictional VP, left the company in August. His June approvals are still good law, and an agent may still follow them. His delegation to the support agent, though, died the day he left. Most people's first instinct is the reverse, which is why one of our evaluation traps is exactly this case.
Routing over a graph that changes
When nobody's precedent covers a request, we escalate to the nearest person who holds the authority right now. algo.SPpaths finds shortest paths, but it has no idea that a reporting line ended last month. Rather than maintain a second "current" edge type, we gave every REPORTS_TO edge a cost: 0 while it is live, 1 once it ends. Then we asked for paths with maxCost: 0:
CALL algo.SPpaths({sourceNode: src, targetNode: cand, relTypes: ['REPORTS_TO'],
weightProp: 'w', costProp: 'dead', maxCost: 0, pathCount: 1})
A ledger that cannot fork
Every decision is appended to a hash chain. Computing the hash in the application meant read-hash-write with retries under contention. Instead we put a pure-JS SHA-256 in the same UDF library and computed the new hash inside the write query:
MATCH (lh:LedgerHead {id:'head'})
...
WITH d, lh, stare.sha256(lh.hash + '|' + $ledgerPayloadHash) AS nh
CREATE (le:LedgerEntry {seq: lh.seq + 1, prev: lh.hash, hash: nh, ...})
SET lh.seq = lh.seq + 1, lh.hash = nh
FalkorDB serialises writes per graph, and a single GRAPH.QUERY is atomic. So 64 concurrent decisions from three agents on 16 connections produce 64 contiguous entries, the shared budget is never overspent, and no case is decided twice. The verifier recomputes the chain with Node's crypto, which is independent of the UDF, and compares every live decision with what the ledger recorded.
Proving it, and being honest about the 100%
We pre-registered an evaluation: 112 requests across 13 kinds of trap. We hashed the answer key and committed the hash before running anything. The key comes from the written rules, checked by a separate TypeScript implementation. That second implementation also has to agree with the UDF on every seeded decision, as a test.
| Correct | False approvals | Refund $ leaked | |
|---|---|---|---|
| Stare | 112/112 | 0 | $0 |
| Vector RAG (top-8 similar decisions + policy + customer directory) | 68/112 | 13 | $14,500 |
| LLM only (policy + customer directory) | 66/112 | 20 | $17,200 |
Stare's score is not the interesting part. The engine executes the rules the key encodes, and the only stochastic piece is the model that turns a message into fields. The baselines are the interesting part. On the Acme-shaped trap ("looks right, rests on an unauthorized approval"), vector RAG scored 0 out of 8, in both runs we kept. On requests that a newer policy forbids, giving the model similar past decisions made it worse than giving it nothing: 6–7 out of 12 against 11–12. The old precedents pulled it towards the old rules. (The baselines are stochastic, so the repo keeps both runs; Stare scored 112/112 in every run.)
What broke
FalkorDB 4.22.1 crashed under our test suite.
- Twice it segfaulted. The first time we were creating indexes and constraints on several graphs at once, from parallel test files.
- The second time we were deleting a few hundred graphs at the start of a run while new ones were being created.
- We now build a schema'd template once, serially, and create every other graph with
GRAPH.COPY. That is also how visitor sandboxes work in production. Each test run starts from a fresh container.
GRAPH.COPY forks the server. Two concurrent copies fail with could not fork, so copies are serialised and retried.
Our erasure scanner passed when it shouldn't have. node.attributes inside a UDF is a host object: its keys don't enumerate, so for (k in attrs) found nothing and every scan came back clean. We caught it only because the erasure certificate is judged against a negative control: a naive "delete the contact node" on a copy that must fail the same checks. It passed, which was impossible, so the scanner was broken. The fix is JSON.parse(JSON.stringify(n.attributes)).
Then the certificate refused a correct erasure. Our synthetic contacts had phone numbers drawn from 100 values and there were 101 contacts. After we erased Marta, her number was still in the graph, on someone else's record. The certificate was right to refuse. Phone numbers are unique now.
Fuzzy matching merged two companies. Jaro-Winkler gives "Acme Logistic GmbH" 0.93 against "Acme Logistics", above our 0.92 threshold. That is a different legal entity (a customer of the other tenant). Now a fuzzy match needs the same number of words and each word to match: typos pass, an extra "GmbH" or "Europe" doesn't.
Read-only ACLs. %R~pattern looks like the obvious read-only key permission. It does not work for graph commands, because their key specs are declared read-write. The read-only auditor (FalkorDB's own MCP server, which we run next to ours) uses ~stare:northwind +GRAPH.RO_QUERY instead. Try a write through it and the database answers NOPERM … 'graph.QUERY'.
MCP 2026-07-28 and the legacy era. People confirm rulings through a multi-round-trip input request (inputRequired.elicit on the server). Our first client test failed with "this 2025-era connection did not declare the required capability". The v2 client defaults to the legacy handshake unless you set versionNegotiation: { mode: 'auto' }.
What we'd tell the next team
- Put the decision where the data is. Once validity was a database function, every surface agreed by construction.
- Make the model's job small. Ours parses fields and writes an explanation. A gate rejects any explanation that names a person, company or amount outside the trace.
- Build the negative control first. It found a bug that every positive test had missed.
Stare is open source (MIT): https://github.com/RaYYeR220/stare. The live desk gives each visitor a private copy of the demo company, where you can overturn a decision, change a policy, erase a customer, or run the 64-way race yourself: https://stare-desk.vercel.app
來源:DEV Community · dev.to