Agent 的資訊洩漏不在檢索，而在生成的句子

The comforting myth goes like this: an agent leaks when it reaches past its permissions. Fix the permissions, fix the leak. So we put ACLs on the vector store, tenant tags on every chunk, row-level security on the SQL tool, and we sleep fine.

I've built that stack. It's necessary and it's not enough, and the reason is structural rather than a bug in my filters.

Here's the shape of the thing. Every document in the leak was individually permitted. The person asking had legitimate access to all of them. Nothing was over-fetched. The disclosure exists only in the composition — in the sentence the agent wrote after reading two documents that were each, on their own, completely fine to read.

Two documents, one sentence

Concrete version, because abstractions hide the problem. Internal support agent, retrieval over the company wiki plus the CRM.

• Doc A: "Pricing changes take effect on the first of the month following the announcement."
• Doc B: "Acme's renewal is booked for the 14th."

Both permitted to the account manager. Both individually boring. The agent, being helpful, writes: "Acme renews two weeks after the new pricing takes effect, so expect an upsell conversation."

That sentence is a leak. Not of either document — of the inference. It's a sales-sensitive claim about a named customer's negotiating position, assembled from two innocuous facts, and it now lives in a ticket that three other people can read. Nobody's ACL was violated. The account manager had every right to both documents. What they didn't have was a grant to have the inference drawn for them and then written into a shared artifact.

This is the mosaic effect, and it's older than LLMs. Intelligence services have compartmented around it for decades. What's new is that we've handed the mosaic assembly to a machine that does it automatically, cheerfully, and at the speed of a token stream.

Why per-document control can't catch it

Access control is a predicate on one object, evaluated at one moment, by one principal. The leak is a property of a set of objects, evaluated after generation, against a sink that may be broader than the conversation. Those are different shapes. You cannot express "this sentence requires two compartments to be true" as a filter on a chunk.

And you can't fix it by tightening the predicate, either. The agent needs both documents to answer the legitimate question. If I strip Doc B because it touches renewal dates, the account manager gets a worse answer to a question they're allowed to ask. Utility and safety are pulling in opposite directions here, which is exactly the sign that you're enforcing at the wrong layer.

It gets worse in three ways that took me a while to appreciate.

Chunks aren't documents. Retrieval hands the model 400-token slices. A slice can contain the qualifying clause that made a sentence safe, or lose it. The unit of permission and the unit of context are not the same unit, and the mismatch is where the sharp edges live.

Agents accumulate. A single-turn RAG lookup is one mosaic. An agent with a session, a scratchpad, and six tool calls is a mosaic being assembled incrementally, and the dangerous combination may span turn two and turn nine with nothing in between that looks suspicious.

The model will not save you. It has no idea who's asking. The system prompt says "you are a helpful assistant with access to the knowledge base," and it is doing precisely that. Asking it to "be careful about sensitive combinations" is asking it to run an access-control policy it cannot see the inputs to. I've tried. It degrades into either useless vagueness or confident silence, and it does so non-deterministically, which is worse than either.

Move the check to the draft

If the leak is a property of the sentence, check the sentence.

The question I want answered, per sentence in the drafted answer: does this sentence require two clearances to be true? Not "did the retrieval stay in bounds" — it did. Not "is this document allowed" — it is. The question is whether the truth of this specific claim depends on facts that live in different compartments, where the combination itself was never granted to anyone.

Rough pipeline I've been prototyping:

1. Label chunks with compartments, not just ACLs. An ACL says who may read this. A compartment says which body of knowledge this belongs to. Two documents can be readable by the same person and still belong to different compartments. That distinction is the whole ballgame and most vector stores don't have a field for it.

2. Force attribution. Make the model cite chunk IDs per sentence, or run a cheap entailment pass afterwards to recover the support set. Attribution is unreliable and I won't pretend otherwise, but "which chunks does this sentence lean on" is a much easier question than "is this sentence safe."

3. Compute the compartment set per sentence. Union of the labels of its supporting chunks. If that union spans more than one compartment, the sentence is a candidate mosaic.

4. Grade the sink. The same sentence in a private chat reply is a smaller event than the same sentence written to a shared ticket, a Slack channel, or an outbound email. The policy should care where the text is going, not just what it says. This is the part I think most teams will skip and most teams will regret skipping.

5. Then decide: generalize, redact, or refuse. Usually the right move isn't blocking. It's "Acme's renewal timing may be affected by the pricing change — check with the account owner." Same information, no inference, points at a human who's allowed to make the call.

The cheap version

Full attribution on every sentence is expensive and I haven't run it at scale, so take this with the appropriate salt. But there's a shortcut that gets most of the value: track which compartments entered the context window during the session. Most sessions never cross one. When a second compartment shows up, flip on the strict pass for the rest of the session. You pay the cost on the small fraction of conversations that are actually risky, and you leave the boring ones alone.

The other thing I'd do tomorrow, and should have done a year ago: log the drafted sentence alongside its support set. My current logs tell me which documents the agent pulled. They tell me nothing about what it concluded. When someone eventually asks "how did this get into the ticket," the retrieval log is not the artifact I need.

The uncomfortable part

I don't think this is fully solvable. Composition risk is inherent to any system that reads broadly and writes fluently, and the honest framing is risk reduction, not prevention. What I do think is that per-document access control has been quietly doing the job of a security theater prop for agent systems — it looks like the control, it satisfies the review, and it is aimed at a different failure than the one that will actually bite you.

The check has to live on the answer. That's the whole post.