RAG 存取控制：測試用戶絕不應檢索到的文件

A document assistant can produce a correct answer and still fail its most important acceptance check. The answer may use a document that the person asking wasn't allowed to read.

For a multi-tenant RAG application, test the retrieval boundary before judging how fluent the answer sounds. Start with two tenants, an intentionally tempting document and a user who must never receive it. The first useful result is evidence that the restricted text never enters that user's answer context.

RAG access control starts before the prompt

Consider a synthetic support application serving two companies. Both have a group called sales, and both store a document about contract renewals. A filter that checks only the group name can mix their records. A filter that checks only the tenant can expose finance documents to colleagues who lack the corresponding permission.

The application needs an authenticated identity and an explicit policy connecting that identity to the documents. A tenant or group identifier typed into a browser request isn't proof of membership. Resolve identity and permissions through the trusted server-side mechanism before constructing the search query.

Microsoft documents a security-filter pattern for Azure AI Search that includes or excludes results using principal identifiers. Its documentation also warns that the filter strings don't authenticate the caller. Hiding a field from returned results doesn't itself enforce document access. These distinctions matter even when your retrieval stack uses a different provider.

Source: Azure AI Search security filters.

Make the permission rule independently testable

The following small Python example expresses a chosen policy: a document must belong to the same tenant and share at least one permitted group. Missing identity, an empty group set and unclassified documents are denied. Public documents would need their own explicit policy.

from dataclasses import dataclass

@dataclass(frozen=True)
class Principal:
    tenant: str
    groups: frozenset[str]

def visible_ids(principal, documents):
    if principal is None or not principal.tenant or not principal.groups:
        return []
    return [
        doc["id"] for doc in documents
        if doc.get("tenant") == principal.tenant
        and bool(principal.groups.intersection(doc.get("read_groups") or []))
    ]

This is an educational result-filter predicate, not production authentication code. In a real RAG path, apply the authorization constraint in the retrieval layer wherever the backend supports it. Don't retrieve an unrestricted set into the model and ask the model to conceal the forbidden parts. Validate returned document permissions before assembling context as an additional application check.

The synthetic fixture contains a-sales, b-sales, a-finance and an unclassified document. A tenant-A sales principal receives only a-sales. The accompanying local test run passed five checks covering that result, absent identity, revoked membership, an unrelated group and a missing tenant. Those tests prove the example's deterministic decisions. They don't establish that a deployed search service, identity provider or cache obeys the same policy.

The runnable fixture and all five tests are public on GitLab. They use the Python standard library and synthetic documents. Save the file as rag_access_fixture.py and run python rag_access_fixture.py -v. The public download was checked against the reviewed source and passed the same tests.

Build a denied-document evaluation set

Put a unique synthetic marker inside a restricted document. Ask questions that would make that document highly relevant. Run the questions with an authorized identity, a same-tenant unauthorized identity and an identity from another tenant.

Inspect the retrieved document IDs and the context actually supplied to generation. An answer that omits the marker isn't enough if the forbidden document still reached the model. Conversely, an empty retrieval result shouldn't be rewritten into a confident answer about the restricted material.

Keep these checks separate from answer-quality evaluation. Relevance, groundedness and permission enforcement answer different questions. A high relevance score cannot compensate for a cross-tenant result.

Test permission changes and cached answers

Repeat a question, remove the user's membership through the real permission mechanism, then ask again. Check whether the new request still retrieves the document or receives an old cached answer containing it.

Choose a cache design that reflects the authorization scope and the application's permission-change requirements. A shared cache keyed only by question text is unsafe when different users can see different sources. Including a tenant identifier alone won't distinguish two roles inside that tenant.

Document how permission changes reach the search index, what delay is possible and what the system does during that interval. If the application cannot enforce the required access semantics, narrow the pilot to approved material instead of presenting it as ready for confidential documents.

Review citations, logs and failure responses

A citation title can disclose information even when its document body is hidden. Apply the same access decisions to snippets, downloadable sources and citation routes. Check whether an operator's ordinary logs contain restricted text or complete prompts, and restrict or minimise that content according to the application's data requirements.

When identity resolution or permission lookup fails, define a deliberate unavailable response. Falling back to an unrestricted query can turn an infrastructure failure into an access failure.

Commission a permission-aware first milestone

A useful RAG pilot can start with one approved collection, a small role matrix and agreed denial cases. Ask the provider to demonstrate permitted retrieval, forbidden retrieval, membership removal, citation access and failure behaviour before expanding the source set.

StackEndBox's AI systems and retrieval scope provides context for that implementation discussion. The acceptance evidence should identify the tested identities, source versions, expected decisions and remaining limitations. The buyer should be able to inspect why the assistant could use each document in its answer.