如何在 Lambda 上用 Fastify 串流 Claude tokens：簡明分步教學

LLM developers assume they need a heavyweight framework to handle streaming responses, but Fastify’s tiny core can proxy Claude’s token stream with almost zero overhead. In this post you’ll see how to turn a Fastify service into a real‑time chat endpoint that streams tokens directly to the browser.

Why Fastify Beats Express for LLM Streaming

When you ask a language model (LLM) like Claude to generate text, it often returns tokens (the smallest pieces of language) one by one. To give a user a “live‑typing” feel you need to forward each token to the browser as soon as it arrives.

Express—the long‑time favorite of Node developers—adds a lot of middleware layers. Those layers are great for flexibility, but each extra function call adds a tiny amount of latency. When you are forwarding dozens of tokens per second, that latency compounds and you can notice a pause between the model’s output and what the user sees.

Fastify is built around performance first principles:

• Minimal core – only the HTTP server and routing logic.
• Built‑in support for async/await – no need for callbacks that can block the event loop.
• Fast request/response handling – uses the ultra‑fast ultra‑http library under the hood.

Think of Fastify as a high‑speed conveyor belt that moves a package (the token) from the model to the client with almost no friction, whereas Express is more like a busy hallway where the same package has to stop at several doors before continuing. For a streaming chat endpoint, that difference is noticeable.

In plain English – Fastify’s thin design means less time waiting inside the server, so the user sees tokens almost as soon as Claude produces them.

Code snippet: a minimal Fastify server

// src/server.ts
import Fastify from "fastify";

// Create a Fastify instance with the smallest possible options
const fastify = Fastify({
  logger: true,               // helpful logs, can be turned off later
  // No plugins loaded – we keep the core tiny
});

fastify.get("/", async (request, reply) => {
  // A simple health‑check endpoint
  return { status: "ok" };
});

// Export the server so Lambda can import it later
export default fastify;

• Fastify() creates the server.
• logger: true prints request details – useful while developing.
• No extra middleware is loaded, keeping the overhead at a few microseconds per request.

Setting Up Fastify with Native fetch on Node.js 22

Node.js 22 ships with a native fetch implementation that works like the browser’s fetch. That means we can call AWS Bedrock (the service that hosts Claude) without pulling in a heavy HTTP client library. Using native fetch keeps the bundle size tiny, which is important for Lambda where every kilobyte counts toward cold‑start time.

Why avoid extra libraries?

Every additional dependency must be zipped, uploaded, and unpacked when Lambda starts. The more code you ship, the longer the cold start (the first time your function runs after being idle). Native fetch eliminates that extra work.

Code snippet: enabling fetch in a Fastify route

// src/routes/chat.ts
import { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";

export async function chatRoute(fastify: FastifyInstance) {
  // Register a POST /chat endpoint
  fastify.post("/chat", async (request: FastifyRequest, reply: FastifyReply) => {
    // Parse the incoming JSON body { prompt: "..." }
    const { prompt } = request.body as { prompt: string };
    if (!prompt) {
      reply.status(400).send({ error: "Prompt is required" });
      return;
    }

    // Call the helper that talks to Bedrock (defined later)
    const stream = await invokeClaudeStream(prompt);

    // Set up Server‑Sent Events (SSE) headers – see next section for details
    reply.raw.writeHead(200, {
      "Content-Type": "text/event-stream", // tells browser to treat data as a stream
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
    });

    // Pipe each token from Bedrock to the client as an SSE line
    for await (const token of stream) {
      // SSE format: each message starts with "data:" and ends with two newlines
      reply.raw.write(`data: ${JSON.stringify({ token })}\n\n`);
    }

    // Close the SSE stream when Bedrock finishes
    reply.raw.end();
  });
}

• request.body holds the JSON payload the browser sends.
• invokeClaudeStream (implemented later) returns an async iterable of tokens – a convenient way to handle a stream in JavaScript.
• reply.raw gives us access to the underlying Node http.ServerResponse so we can write raw SSE data.

Tip – Keep fetch calls and response handling inside the route as short as possible. Anything that can be moved to a helper function reduces the amount of code that runs during each request, making the Lambda cold start faster.

Calling Claude via Bedrock and Streaming SSE

Bedrock is Amazon’s managed service that hosts LLMs like Claude. The SDK package we use is @aws-sdk/client-bedrock-runtime. Inside the SDK there is a method called InvokeModelCommand that can be asked to stream the model’s output instead of waiting for the whole answer.

What is “streaming” in this context?

Imagine Claude is a storyteller reading a book aloud. With normal (non‑streaming) calls, you wait until the storyteller finishes the whole chapter before you hear anything. With streaming, you hear each sentence as soon as it is spoken. In technical terms, the service sends a chunked HTTP response where each chunk contains the next token.

Setting up the Bedrock client

// src/bedrock.ts
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";
import { Readable } from "stream";

// Create a client that uses the default Lambda credentials
const bedrock = new BedrockRuntimeClient({});

// Helper that returns an async iterable of tokens
export async function* invokeClaudeStream(prompt: string): AsyncGenerator<string> {
  // Build the request payload according to Claude's API spec
  const body = JSON.stringify({
    prompt,
    // Tell Claude we want a streaming response
    stream: true,
    // Optional: max tokens, temperature, etc.
  });

  // The command carries the model ID and the payload
  const command = new InvokeModelCommand({
    modelId: "anthropic.claude-v2", // replace with the exact model you use
    contentType: "application/json",
    accept: "application/json",
    body,
  });

  // Invoke the model – Bedrock returns a Readable stream of bytes
  const response = await bedrock.send(command);

  // The response.body is a Node.js Readable stream; we convert it to async iterator
  const stream = response.body as Readable;

  // The raw bytes are newline‑delimited JSON fragments; each contains a token
  const decoder = new TextDecoder("utf-8");
  let buffer = "";

  for await (const chunk of stream) {
    // Convert Uint8Array chunk to string and accumulate
    buffer += decoder.decode(chunk, { stream: true });

    // Split on newline – each line is a complete JSON message
    const lines = buffer.split("\n");
    // Keep the last partial line in buffer for the next iteration
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (!line.trim()) continue; // skip empty lines
      const data = JSON.parse(line);
      // The token appears under `completion` in Claude's streaming format
      yield data.completion;
    }
  }

  // After the stream ends, flush any remaining buffered data
  if (buffer) {
    const data = JSON.parse(buffer);
    yield data.completion;
  }
}

• BedrockRuntimeClient reads credentials from the Lambda execution role, so we don’t embed keys.
• InvokeModelCommand tells Bedrock which model to run and that we want a streaming response (stream: true).
• The function invokeClaudeStream is an async generator – it yields each token as soon as it arrives, letting the Fastify route for await over it.
• The newline‑delimited JSON format is a common pattern for streaming LLMs; each line is a small JSON object that includes a completion field containing the next token.

In plain English – We ask Bedrock to talk to us one word at a time, turn each word into a JavaScript value, and hand it over to Fastify so the browser can see it instantly.

Handling backpressure

Backpressure is a fancy way of saying “don’t overwhelm the client”. When the browser can’t read data quickly enough, Node’s stream will automatically pause reading from Bedrock until the socket drains. Because we use for await on the async generator, the await implicitly respects that pause, so we don’t need extra flow‑control code.

Deploying the Fastify Service to Lambda

AWS Lambda runs your code inside a sandboxed container. By default, Lambda buffers the whole HTTP response before sending it to the caller. That behavior defeats streaming – the client will only get the final payload after the function finishes. To make streaming work we must:

1. Turn off buffering – set the Content-Type to text/event-stream and add the header X-Amz-Content-Type with the same value.
2. Return a “chunked” response – don’t call reply.send() after writing data; instead write directly to reply.raw and end the response when the stream finishes.
3. Set callbackWaitsForEmptyEventLoop = false – tells Lambda to finish the invocation as soon as the handler returns, even if the event loop still has pending work (our open HTTP connection).

Lambda handler file

// src/lambda.ts
import fastify from "./server";
import { chatRoute } from "./routes/chat";

// Register the chat route on the shared Fastify instance
await chatRoute(fastify);

// Export the handler that Lambda will call
export const handler = async (event: any, context: any) => {
  // Prevent Lambda from waiting for the event loop to be empty
  context.callbackWaitsForEmptyEventLoop = false;

  // Fastify can handle raw API Gateway events via the "aws-lambda-fastify" shim.
  // We avoid an extra package by using Fastify’s built‑in `server.inject`.
  const result = await fastify.inject({
    method: event.httpMethod,
    url: event.path,
    headers: event.headers,
    payload: event.body,
    // The `raw` field gives us direct access to the Node response object
    // which Fastify already wrote to in the /chat route.
  });

  // Return the shape API Gateway expects
  return {
    statusCode: result.statusCode,
    headers: result.headers,
    // For streaming we must return the raw body stream; API Gateway v2 supports this.
    // If you use the older REST API, you’ll need to enable “Lambda proxy integration”.
    body: result.body,
    isBase64Encoded: false,
  };
};

• context.callbackWaitsForEmptyEventLoop = false; disables the default buffering behavior.
• fastify.inject simulates an HTTP request without opening a network socket, which works nicely inside Lambda because we already have the request data from API Gateway.
• The returned object matches the Lambda proxy integration format that API Gateway expects.

Deploy script (using the AWS CDK for illustration)

// infra/cdk-stack.ts
import * as cdk from "aws-cdk-lib";
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as apigateway from "aws-cdk-lib/aws-apigatewayv2";

export class ClaudeStreamingStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string) {
    super(scope, id);

    const fn = new lambda.Function(this, "FastifyClaudeFn", {
      runtime: lambda.Runtime.NODEJS_22_X,
      handler: "src/lambda.handler",
      code: lambda.Code.fromAsset("dist"), // compiled TypeScript output
      memorySize: 1024, // give enough RAM for Bedrock client
      timeout: cdk.Duration.seconds(30),
      environment: {
        // Bedrock region, etc.
        AWS_REGION: "us-east-1",
      },
    });

    // API Gateway HTTP API (v2) with Lambda integration
    const api = new apigateway.HttpApi(this, "ClaudeApi", {
      createDefaultStage: true,
    });

    api.addRoutes({
      path: "/chat",
      methods: [apigateway.HttpMethod.POST],
      integration: new apigateway.LambdaProxyIntegration({ handler: fn }),
    });
  }
}

Gotcha – Lambda’s response streaming works only with the HTTP API (v2), not the older REST API. Forgetting this will make the browser receive a single JSON blob instead of a live token stream.

Lambda layer gotcha reminder

If you decide to add a custom layer (e.g., for shared utilities) note that Node 22’s ESM loader (require(esm)) can silently break when the layer is compiled for CommonJS. The safest route is to keep everything in native ESM (.mjs or "type":"module" in package.json) or avoid the esm shim entirely.

Testing the End‑to‑End Token Stream

Testing streaming is a little different from a normal API call because you need to read data as it arrives. The easiest way is to use curl with the --no-buffer flag or a small Node script that consumes the SSE stream.

Using curl

curl -N -X POST https://<api-id>.execute-api.<region>.amazonaws.com/chat \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Tell me a short story about a robot."}'

• -N disables curl’s internal buffering, letting you see each data: line instantly.
• You should see lines like:

data: {"token":"Once"}
data: {"token":" upon"}
data: {"token":" a"}
...

Node test script

// test/stream-client.js
import fetch from "node-fetch";

async function run() {
  const response = await fetch("https://<api-id>.execute-api.<region>.amazonaws.com/chat", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt: "Explain quantum computing in one sentence." }),
  });

  // Ensure we get an SSE stream
  if (!response.ok || !response.headers.get("content-type")?.includes("text/event-stream")) {
    console.error("Not a streaming response");
    return;
  }

  // Read the stream line‑by‑line
  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (line.startsWith("data:")) {
        const payload = JSON.parse(line.slice(5).trim());
        process.stdout.write(payload.token);
      }
    }
  }

  console.log("\n--- stream finished ---");
}

run();

• The script fetches the /chat endpoint, checks that the Content-Type is the SSE type, and then reads each chunk as it arrives.
• process.stdout.write prints tokens without a newline, mimicking the “live typing” effect in a terminal.

Tip – When you test locally (e.g., npm run dev with Fastify listening on localhost), you can skip the Lambda layer and call the route directly. The same SSE headers work, so the client code stays identical.

Verifying Lambda buffering is disabled

If you forget to set callbackWaitsForEmptyEventLoop = false, the client will receive nothing until the model finishes. You’ll see a single huge JSON payload instead of many data: lines. Open the CloudWatch logs for your Lambda – you’ll notice a line like REPORT RequestId: … Duration: … Billed Duration: … after the entire response is sent. Once the flag is set, the log will show the function ending while the connection remains open, confirming streaming works.

The Takeaway

What you now know

• Fastify’s lean core makes it a perfect match for token‑by‑token streaming; less middleware means lower latency.
• Native fetch in Node 22 removes the need for extra HTTP libraries, shrinking Lambda bundle size and cold‑start time.
• Bedrock’s InvokeModelCommand with stream: true returns a newline‑delimited JSON stream; turning that into an async generator gives a clean for await loop.
• Lambda buffers responses unless you disable it (callbackWaitsForEmptyEventLoop = false) and use the HTTP API with text/event-stream headers.
• Backpressure is handled automatically when you await each token; you don’t need manual flow‑control code.
• Common pitfalls: using the old REST API (no streaming), forgetting the SSE headers, and pulling in an esm shim that breaks under Node 22.

Armed with these pieces, you can ship a production‑grade, real‑time chat endpoint that feels as responsive as typing in a messenger, all while staying within the modest cost envelope of a single Lambda function. Happy streaming!

Transparency notice

This article was written with the help of an AI system — Groq (GPT OSS 120B).

Published: 2026-11-04 · Primary focus: Fastify

All code blocks are intended to be correct and runnable, but please verify them
against the official docs for the tools mentioned before using in production.

Find an error? Drop a comment — corrections are always welcome.