上下文優化不等於串流恢復：整合 context-mode 時避免掩蓋故障

Your production agent fails at 3:47 AM. The on-call engineer opens the transcript: thirty-two thousand lines of test output, followed by a cheerful "I've analyzed the failures and here's my recommendation." The model never saw the actual error. It hit the context ceiling three tool calls ago and has been operating on stale summaries since.

This is not a conversation-history problem. A five-turn chat consumes maybe 8,000 tokens. A single pytest run with verbose output can return 50,000. One file search across a monorepo can yield 200 results. The agent keeps working, the streaming UI keeps typing, and nothing alerts you that the decisions being made are now based on truncated evidence.

mksglu/context-mode attacks that specific failure mode by sandboxing tool output, enforcing routing through MCP hooks, and persisting session memory outside the immediate inference context. The project reports 98% output reduction across 17 platforms. I have not reproduced those measurements.

As an external integrator, I evaluated one narrower question: Can a tool-output boundary reduce model input without making streamed responses misleading when the transport fails?

That requires keeping context management and stream delivery architecturally separate.

Put the reduction boundary before inference

The architecture splits into two paths.

The agent execution path routes tools through context-mode's supported mechanisms. Large outputs stay outside the model's immediate context; the model receives bounded, semantically selected information. Session memory maintains continuity across turns.

The presentation path takes the resulting conversation, invokes inference, and streams assistant output into React using the Vercel AI SDK.

Do not insert a generic truncator between those paths. Cutting a pytest log at 8,000 characters preserves setup noise while removing the actual failure line. Cutting serialized messages can corrupt tool-call relationships, leaving the model with a reference to a result that no longer exists in context.

Instead, enforce an application-level contract: the conversation submitted for inference must already contain bounded, relevant tool results. Preserve a reference to complete output wherever the execution environment supports later retrieval. Context-mode's configuration and retrieval semantics are deployment-specific; they are not interchangeable with AI SDK transport settings.

This matters because smaller input does not make delivery reliable. A proxy can still buffer output, a browser can disappear, and a user can cancel mid-sentence.

A bounded streaming route

The following Next.js App Router handler is the inference boundary, not a replacement for context-mode's MCP integration.

Install the compatible AI SDK 5 family: ai@5, @ai-sdk/openai@2, and @ai-sdk/react@2 for the client. Commit the lockfile. Set B_LOST_API_KEY and MODEL_ID in the server environment. The model identifier must be supported by your configured endpoint.

B-Lost's OpenAI-compatible relay at https://b-lost.com/v1 is the gateway in this topology. Whether its complete deployment path delivers unbuffered SSE is something to measure, not assume.

// app/api/chat/route.ts
import {
  convertToModelMessages,
  streamText,
  validateUIMessages,
} from "ai";
import { createOpenAI } from "@ai-sdk/openai";

export const runtime = "nodejs";
export const maxDuration = 60;

const MAX_INPUT_BYTES = 120_000;

export async function POST(request: Request) {
  const apiKey = process.env.B_LOST_API_KEY;
  const modelId = process.env.MODEL_ID;

  if (!apiKey || !modelId) {
    return Response.json(
      { error: "Inference configuration is missing." },
      { status: 503 },
    );
  }

  // Also enforce a body-size limit at the ingress proxy.
  const body = await request.text();

  if (new TextEncoder().encode(body).byteLength > MAX_INPUT_BYTES) {
    return Response.json(
      { error: "Reduce tool output before requesting inference." },
      { status: 413 },
    );
  }

  let messages;
  try {
    const payload = JSON.parse(body);
    messages = await validateUIMessages({
      messages: payload.messages,
    });
  } catch {
    return Response.json(
      { error: "Invalid UI message payload." },
      { status: 400 },
    );
  }

  const provider = createOpenAI({
    apiKey,
    baseURL: "https://b-lost.com/v1",
  });

  try {
    const result = streamText({
      model: provider.chat(modelId),
      messages: await convertToModelMessages(messages),
      abortSignal: AbortSignal.any([
        request.signal,
        AbortSignal.timeout(55_000),
      ]),
      maxRetries: 0,
      maxOutputTokens: 2048,
    });

    return result.toUIMessageStreamResponse({
      headers: {
        "Cache-Control": "no-cache, no-transform",
        "X-Accel-Buffering": "no",
      },
      onError: () => "Generation interrupted. Retry explicitly.",
    });
  } catch {
    return Response.json(
      { error: "Unable to start generation." },
      { status: 502 },
    );
  }
}

The byte limit is an admission guard, not a token budget. It runs after reading the body, so ingress must enforce its own limit. Neither this guard nor message validation constitutes authentication or tenant authorization.

The 55-second deadline leaves headroom below the route's 60-second deployment allowance. Platforms impose different limits. The abort signal requests cancellation; propagation of browser disconnects through hosting infrastructure still needs testing.

Diagnose the boundary that actually failed

Use useChat from @ai-sdk/react with its default transport pointed at /api/chat. This route emits the AI SDK UI-message stream protocol. A raw OpenAI SSE parser is the wrong consumer.

That distinction explains a common integration failure: the network tab shows incoming bytes, but the UI never updates because the client expects a different event format. Do not manually split arbitrary network chunks on newlines; chunk boundaries need not match SSE frames or UTF-8 character boundaries.

For buffering diagnostics, compare upstream first-byte time, route first-byte time, and the browser's first rendered text. Run curl -N against the deployed route with a valid message payload. If curl receives periodic events but React updates in bursts, investigate rendering pressure. With the AI SDK 5 React hook, experimental_throttle can reduce update frequency; rendering expensive Markdown on every delta can otherwise dominate the main thread.

If both curl and the browser receive one final burst, investigate the relay and hosting path. X-Accel-Buffering: no is advisory, not a universal buffering override.

Keep interruption visible

The deliberate trade-off here is rejecting oversized conversations rather than silently degrading them. The agent must return to its context-management layer and select better evidence.

Disabling automatic retries avoids treating a second generation as continuation of the first. It sacrifices some transient-error recovery for predictable UI semantics.

After cancellation or transport failure, keep partial assistant text visibly incomplete. Do not promote it into persistent session memory as a completed conclusion. Context-mode can reduce what enters inference; the application still owns whether an interrupted answer becomes durable agent history.

The hardest operational dilemma: you can reduce what enters the model, or you can retry failed streams automatically, but doing both without losing signal fidelity requires explicit session-state reconciliation that most frameworks do not provide. When your agent's context budget runs out mid-task, do you surface the failure to the user, retry with reduced context and risk nondeterministic behavior, or persist partial work and rely on the next turn to recover? How are you handling context exhaustion and stream interruption in production agent systems?