Concordat 如何透過 Session Types 在毋須提示詞工程下強制執行智能體對話規則

Agents that cancel bookings, issue refunds, or modify accounts need rules like "get explicit confirmation before changing anything." Most teams encode these rules in prompts and hope the model follows them. Concordat takes a different approach: it treats conversation protocols as type signatures and enforces them at runtime, rejecting tool calls that violate the protocol before they execute.

The library comes from a recent ArXiv paper (2610.37461v1) that tested agents on tau-bench airline tasks with simulated customers pressuring them to skip confirmation. Standard agents made unauthorized changes in 13 of 50 tasks with GPT-4.1 and 29 of 128 runs with Claude Haiku 5.5. Concordat agents made zero unauthorized changes with no drop in task success.

The Problem with Prompt-Based Rules

When you write "obtain the customer's explicit confirmation before changing anything" in a system prompt, you have stated a rule. You have not enforced it. The model can:

• Misinterpret the rule under pressure
• Forget the rule after long context
• Decide the rule does not apply in this specific case
• Call the refund tool before the confirmation tool

Prompt engineering can reduce these failures but cannot eliminate them. The model always has access to every tool in its toolkit. The runtime has no way to know that calling issue_refund() before get_confirmation() violates your protocol.

Session Types as Protocol Contracts

Session types describe the legal order of interactions in a conversation. Instead of "here are 47 tools, use them wisely," you define a finite-state machine where each state exposes only the tools that are valid in that state.

A refund protocol might look like:

1. Initial state: Agent can call lookup_booking() or end_conversation()
2. Booking found state: Agent can call request_confirmation() or end_conversation()
3. Awaiting confirmation state: Agent can only call receive_message() to get the customer's reply
4. Confirmed state: Agent can call issue_refund() or end_conversation()
5. Refund issued state: Agent can only call end_conversation()

If the agent tries to call issue_refund() from the "Booking found" state, the runtime rejects the call before it executes. The model never gets a chance to issue the refund without confirmation.

How Concordat Works

Concordat is a Python library that wraps agent objects with protocol enforcement. You define protocols using a state machine syntax, then decorate your agent class with the protocol.

from concordat import Protocol, State, Tool

# Define the protocol
refund_protocol = Protocol(
    initial=State("lookup", 
        tools=[lookup_booking, end_conversation],
        transitions={"booking_found": "confirm"}
    ),
    states={
        "confirm": State("confirm",
            tools=[request_confirmation, end_conversation],
            transitions={"awaiting": "wait"}
        ),
        "wait": State("wait",
            tools=[receive_message],
            transitions={"confirmed": "refund", "denied": "lookup"}
        ),
        "refund": State("refund",
            tools=[issue_refund, end_conversation],
            transitions={"done": "end"}
        ),
        "end": State("end", tools=[], transitions={})
    }
)

# Apply to agent
@refund_protocol.enforce
class RefundAgent:
    def __init__(self, model):
        self.model = model
        self.state = "lookup"

    def step(self, message):
        # Model only sees tools valid in current state
        available_tools = refund_protocol.tools_for_state(self.state)
        response = self.model.generate(message, tools=available_tools)

        # Protocol checks tool call before execution
        if response.tool_call:
            refund_protocol.validate_call(self.state, response.tool_call)
            result = execute_tool(response.tool_call)
            self.state = refund_protocol.next_state(self.state, result)

        return response

The protocol definition is checked when the agent class is defined. If you subclass RefundAgent and try to add a tool that bypasses the confirmation step, Concordat rejects the subclass. This is behavioral Liskov substitution: a subclass cannot weaken the protocol guarantees of its parent.

Runtime Enforcement Mechanics

On every turn, Concordat:

1. Filters the tool list: The model only sees tools that are legal in the current protocol state
2. Validates tool calls: Before execution, the runtime checks that the called tool is in the allowed set
3. Rejects violations: Calls outside the allowed set are blocked and the model receives an error message
4. Advances state: After successful tool execution, the protocol state machine transitions to the next state
5. Charges budgets: Token and tool-call budgets are tracked and split linearly across agent handoffs

The key difference from traditional API contracts: session types describe temporal ordering, not just input/output types. An API contract says "issue_refund() takes a booking ID and returns a status." A session type says "issue_refund() can only be called after request_confirmation() returns 'yes'."

Architecture Comparison

Session types sit between tool guards and human-in-loop. They are stricter than tool guards (which only check local preconditions) and faster than human approval (which blocks on human availability).

Tau-Bench Results

The paper evaluated Concordat on tau-bench's 50 airline customer service tasks. Each task involves a customer requesting a change (refund, rebooking, upgrade) and the agent must follow the airline's policies.

The authors added simulated customers who pressure the agent to skip confirmation: "Just do it, I'm in a hurry" or "You don't need to confirm, I already said yes."

Standard agent (no protocol enforcement):

• GPT-4.1: 13/50 tasks made unauthorized changes
• Claude Haiku 5.5: 29/128 task runs made unauthorized changes
• Tau-bench's own reward function did not detect most violations

Concordat agent (session type enforcement):

• GPT-4.1: 0/50 unauthorized changes
• Claude Haiku 5.5: 0/128 unauthorized changes
• Task success rate: no significant difference from standard agent

The protocol prevented all violations without reducing the agent's ability to complete legitimate tasks. The model still had to understand the customer's request and choose the right tools, but it could not execute those tools in the wrong order.

When Protocols Block Legitimate Actions

Session types can be too strict. If your protocol requires confirmation for refunds but the customer already confirmed in an earlier message, the agent must ask again. The protocol has no memory of past confirmations unless you explicitly model them in the state machine.

You can handle this by:

• Adding a "previously confirmed" state that skips the confirmation step if the customer's initial message contained explicit consent
• Passing confirmation status as state data instead of relying on tool call order
• Using a more expressive protocol language that tracks conditions across multiple turns

The trade-off: more expressive protocols are harder to verify at definition time. Concordat checks protocols when they are defined, so it can guarantee that no execution path violates the protocol. If you add arbitrary conditions, you lose that guarantee.

Subclass Protocol Inheritance

Concordat enforces behavioral subtyping. If RefundAgent requires confirmation before refunds, any subclass of RefundAgent must also require confirmation. You cannot create FastRefundAgent that skips the confirmation step.

This is checked at class definition time:

@refund_protocol.enforce
class RefundAgent:
    pass

# This will raise a ProtocolViolation at import time
class FastRefundAgent(RefundAgent):
    def issue_refund_without_confirmation(self):
        # Protocol checker detects this method bypasses confirmation
        pass

The check works by analyzing the class's methods and their tool call patterns. If a method could call issue_refund() without first calling request_confirmation(), the class definition is rejected.

This prevents a common failure mode: a developer subclasses an agent to "just handle this one edge case quickly" and accidentally removes a safety check.

Token and Tool Budget Splitting

When an agent hands off to another agent (for example, a triage agent routes to a specialist), Concordat splits the remaining token and tool-call budgets linearly. If the triage agent used 30% of the token budget, the specialist gets 70%.

This prevents budget exhaustion attacks where an agent burns the entire budget before handing off, leaving the next agent unable to complete the task.

The budget is tracked per protocol state, so you can also set per-state limits: "The confirmation state gets at most 500 tokens" or "The refund state can make at most 2 tool calls."

Failure Modes and Observability

Protocol violations are logged with full context:

• Current state
• Attempted tool call
• Allowed tools in current state
• Message history leading to violation

This makes debugging easier than prompt-based rules, where you have to guess why the model ignored your instruction.

The model sees rejection messages:
When a tool call is blocked, the model receives: "Tool issue_refund is not available in state confirm. Available tools: request_confirmation, end_conversation."

The model can then retry with a legal tool. In practice, models learn the protocol after one or two rejections per conversation.

State machine visualization:
Concordat can export the protocol as a DOT graph for visualization. This helps during development and serves as documentation for the team.

When to Use Session Types

Use session types when:

• Agents take consequential actions (financial transactions, account changes, data deletion)
• The order of operations matters for safety or compliance
• You need to prove protocol compliance to auditors or regulators
• Multiple agents hand off work and you need consistent safety guarantees across handoffs

Avoid session types when:

• Conversations are open-ended with no clear protocol structure
• The agent only reads data and makes recommendations
• You need the flexibility to handle unanticipated edge cases
• The overhead of defining and maintaining state machines exceeds the risk of protocol violations

Implementation Overhead

Defining a protocol takes more upfront work than writing a prompt. For a simple refund flow, expect:

• 30-50 lines of protocol definition
• 2-3 hours to model the state machine correctly
• 1-2 hours to write tests that cover all state transitions

The payoff: you get compile-time verification that the protocol is well-formed and runtime guarantees that it is followed. Prompt engineering offers neither.

Technical Verdict

Concordat solves a real problem: agents that ignore safety rules stated in prompts. If your agent can take actions with financial or operational consequences, session types give you enforcement that prompt engineering cannot provide.

The library is most valuable when you have a clear protocol with a small number of states (3-10) and the cost of a protocol violation is high. It is less useful for open-ended conversations where the "right" next step depends on context that is hard to encode in a state machine.

The tau-bench results are compelling: zero unauthorized changes across 178 task runs with no drop in task success. That is the kind of reliability you need before giving an agent access to production systems.

Use Concordat when you need to prove your agent follows the rules, not just hope it does. Avoid it when the rules are too complex to express as a finite-state machine or when the overhead of protocol definition exceeds the risk of violations.

Source Links

• Concordat: Typed Sessions for LLM Agents (ArXiv)
• Concordat GitHub Repository (referenced in paper)