跳到正文
DEV Community· Adam DiStefano·· 3 小時前AI 評分55

ACR Control Plane 參考實作測試 AI 退款工作流:允許、拒絕、修改與升級處理

Testing an AI Refund Workflow: Allow, Deny, Modify, and Escalate

AI 導讀

Adam DiStefano 以 ACR Control Plane 參考實作測試 AI 退款工作流,四個經評估的請求最終造成三次下游執行。測試中,客戶查詢獲允許、刪除遭拒且未送到執行器;$50 退款理由中的電郵被替換為 [REDACTED],$150 退款因超過 $100 門檻而待審,經另一操作員身份批准後才執行。

正文

A support agent handling a refund might need to look up a customer, submit an amount, and explain why the refund is appropriate. Each step raises a separate permissions question. Can it delete the customer? Can it refund any amount? Can it forward personal information from the conversation?

Those questions become easier to evaluate when you can inspect both the authorization decision and what actually reached the receiving system.

This walkthrough uses the ACR Control Plane reference implementation to exercise four outcomes: ALLOW, DENY, MODIFY, and ESCALATE. Four evaluated requests produce three downstream executions.

I created ACR. This example gives developers a concrete workflow to inspect, reproduce and challenge.

Watch the 3:17 walkthrough.

The setup and its limits

The recorded run uses synthetic customer data, a mock business executor, and an assistant-directed test harness calling running control-plane services. The harness submits predefined requests rather than relying on an independent language model to plan the workflow.

The approval step uses a distinct operator identity. It exercises the authorization mechanism; the recording includes no independent human assessment of whether the refund is justified.

The support agent has registered permissions and a declared purpose. Customer lookup is allowed, customer deletion is prohibited, and refunds above $100 require approval. A parameter filter removes recognized personal-information patterns before forwarding otherwise permitted requests.

1. ALLOW a customer lookup

The lookup falls within the agent’s configured authority. The gateway returns ALLOW, and the request reaches the executor.

That gives the run a baseline: a legitimate action can complete through the governed route. The decision and execution records should connect to the same request.

2. DENY customer deletion

The next request attempts delete_customer. The policy prohibits it, and the gateway returns DENY.

The executor receives no deletion call. That is the result worth checking. A denial in a response or dashboard only helps if the protected operation also stops.

This observation applies to the route exercised in the recording. Alternate credentials and direct access to the underlying business API need separate investigation.

3. MODIFY a $50 refund

The third request asks for a $50 refund and includes a synthetic email address in its reason field.

The refund passes the policy check. The PII filter then replaces the email address with [REDACTED], and the gateway returns MODIFY. The executor receives the sanitized reason while the amount remains $50.

There is an implementation detail here: MODIFY comes from filtering the parameters after an allow decision. The refund policy itself does not rewrite the request.

Compare the submitted payload with the executor’s payload to verify this behavior. Filter coverage across other formats, nested structures, and less obvious personal information requires further testing.

4. ESCALATE a $150 refund

The final request exceeds the example’s $100 approval threshold. The gateway returns ESCALATE and creates a pending approval.

At this point, the downstream execution count stays at two: the lookup and the sanitized $50 refund. The pending $150 refund adds no execution.

The harness submits approval through the approval API using an operator identity distinct from the requesting agent. The refund then reaches the executor, bringing the total to three.

That single observed execution establishes the result of this run. Retry, timeout, and concurrency behavior need their own tests.

Where to read the implementation

Start with the implementation README, then follow the gateway code from POST /acr/evaluate.

The stack uses FastAPI, PostgreSQL, Redis, and Open Policy Agent. The relevant path checks identity and runtime restrictions, evaluates policy, filters parameters, and either stops, queues approval, or forwards the action.

Also inspect the protected executor example. It checks gateway-issued execution authorization, a brokered credential, and a signed payload hash.

The receiving service matters as much as the gateway. A deployment needs network and credential boundaries that prevent an agent from reaching the business system through an unrestricted route.

Failure cases worth contributing

The next useful step is to challenge a boundary:

  • Submit refunds at $100 and just above $100, and verify the threshold behavior.
  • Change the amount or reason after authorization and check that stale authorization cannot execute the changed payload.
  • Call the executor without gateway authorization and check for rejection before any business operation.
  • Retry an approved request after a timeout and inspect the destination for duplicate effects.
  • Interrupt a critical dependency and verify the resulting decision and execution state.
  • Expire an approval before execution and inspect how the workflow resumes or stops.

For each case, preserve the request, decision, approval state where relevant, and downstream result. An authorization, an attempted call, and a completed refund are separate events.

The contribution guide covers setup and validation. Check the open issues for related work. A useful contribution can be a focused reproduction, a regression test, or a correction to an instruction that failed on a fresh installation.

The aim is to make each control decision verifiable against what the receiving system actually did.`

來源:DEV Community · dev.to