GitLab 中 DevOps 即時串流架構指南：Server-Sent Events、WebSockets 與 MCP

Architecting Real-Time Streaming in DevOps: Server-Sent Events, WebSockets & MCP in GitLab

Traditional web applications rely on standard request-response HTTP cycles. You submit a form or fetch a list of items, and the server returns a complete payload.

However, in modern DevOps platforms and AI systems like GitLab, static HTTP falls short:

1. Live CI/CD Terminal Output: Developers need to watch compiler logs and test outputs stream line-by-line in real time.
2. AI Code Suggestions & Duo Chat: When an LLM generates a 200-line refactor, waiting 15 seconds for the entire response to buffer feels painfully sluggish compared to streaming tokens instantaneously as they are computed.
3. Model Context Protocol (MCP): Connecting autonomous AI agents (such as Cline, Cursor, or Antigravity) to GitLab requires persistent bidirectional or event-driven streaming transports like Server-Sent Events (SSE) and stdio.

In this architecture guide, we will unpack how real-time streaming works across GitLab’s core infrastructure, comparing WebSockets, Server-Sent Events (SSE), and MCP streaming transports, and show how to implement a high-throughput SSE streaming service.

1. Architectural Comparison: WebSockets vs. SSE vs. MCP Transports

+-------------------------------------------------------------+
|                      WEBSOCKETS (Full-Duplex)               |
|                                                             |
|   Client  <====== Bidirectional TCP Upgrade (ws://) ======> |
|   - Best for: Interactive web terminal consoles (xterm.js). |
|   - Drawback: Complex proxy buffering, corporate firewalls. |
+-------------------------------------------------------------+

                              vs.

+-------------------------------------------------------------+
|                SERVER-SENT EVENTS / SSE (Half-Duplex)       |
|                                                             |
|   Client  <--- Single HTTP/2 Stream (text/event-stream) --- |
|   - Best for: GitLab Duo Chat token streaming, CI logs.     |
|   - Advantage: Native browser reconnects, simple HTTP/2.    |
+-------------------------------------------------------------+

                              vs.

+-------------------------------------------------------------+
|              MODEL CONTEXT PROTOCOL (MCP stdio / SSE)       |
|                                                             |
|   Agent  <=== JSON-RPC 2.0 Packets over stdio or SSE ===>   |
|   - Best for: Autonomous AI coding agents querying GitLab.  |
|   - Advantage: Standardized capability negotiation.         |
+-------------------------------------------------------------+

2. Implementing Server-Sent Events (SSE) for AI Token Streaming

When building streaming services for GitLab Duo or custom developer bots, Server-Sent Events (SSE) is almost always the superior choice over WebSockets:

• It runs over standard HTTP/1.1 or HTTP/2 (Content-Type: text/event-stream).
• Standard HTTP load balancers (NGINX, Cloudflare, Traefik) proxy it seamlessly without special upgrade handshakes.
• Native browser support (EventSource) handles automatic exponential backoff reconnection out of the box.

Building a High-Performance SSE Streaming Handler (FastAPI / Python):

import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

async def generate_token_stream(prompt: str):
    """Simulates real-time token generation from an AI model engine."""
    tokens = [
        "def ", "calculate_metrics", "(data: ", "list):", "\n",
        "    # Fast vector aggregation\n",
        "    return ", "sum(data) ", "/ ", "len(data)\n"
    ]

    for token in tokens:
        # Format payload strictly adhering to the SSE standard: `data: <content>\n\n`
        yield f"data: {token}\n\n"
        await asyncio.sleep(0.04) # Simulate 40ms token generation interval

@app.post("/v1/stream-suggestions")
async def stream_suggestions(payload: dict):
    prompt = payload.get("prompt", "")
    return StreamingResponse(
        generate_token_stream(prompt),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no" # Prevents NGINX from buffering chunked output!
        }
    )

Crucial Production Gotcha: The X-Accel-Buffering: no header tells reverse proxies like NGINX not to buffer responses. Without this header, NGINX will buffer 4KB of output before sending any packets, completely breaking the real-time streaming illusion!

3. How Model Context Protocol (MCP) Utilizes Streaming

In GitLab Core (where we contributed MR !260212 for MCP resources), the Model Context Protocol introduces two standard streaming transports:

1. stdio (Local Process Pipe):

   • When running locally inside an IDE or terminal, the AI client spawns the GitLab MCP server as a sub-process and streams newline-delimited JSON-RPC messages over standard input and output.
   • Zero networking overhead, sub-millisecond local latency.

2. SSE Transport (Remote Web Services):

   • For remote GitLab instances, the client opens a GET request to an /sse endpoint to receive server-sent events.
   • The server replies with an active session ID and endpoint URI, allowing the client to POST JSON-RPC commands while receiving real-time responses over the persistent SSE pipe.

4. Frontend Consumption with React / TypeScript

Connecting a frontend web dashboard to an SSE stream in modern web applications requires minimal boilerplate:

async function fetchStreamingSuggestion(prompt: string, onToken: (text: string) => void) {
  const response = await fetch('/v1/stream-suggestions', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt })
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder('utf-8');

  if (!reader) return;

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    const chunk = decoder.decode(value, { stream: true });
    const lines = chunk.split('\n');

    for (const line of lines) {
      if (line.startsWith('data: ')) {
        const token = line.replace('data: ', '');
        onToken(token);
      }
    }
  }
}

5. Key Architecture Takeaways

1. Prefer SSE for LLMs & Logs: Server-Sent Events provide native HTTP caching, reconnection, and simplicity compared to full-duplex WebSockets.
2. Disable Reverse Proxy Buffering: Always set X-Accel-Buffering: no on your streaming endpoints to prevent NGINX or Traefik from delaying tokens.
3. Embrace Open Protocol Standards: Standardized protocols like MCP ensure your streaming services are instantly compatible with any AI client across the industry.

Resources & Links

• GitLab Profile: gitlab.com/hrlpavan
• GitLab Contributor Profile: contributors.gitlab.com/users/hrlpavan
• GitHub: @hrlpavan

How does your team handle real-time streaming in your internal developer tools? Let’s connect in the comments below!