Mastra Agent 接入自訂 OpenAI 相容 API：url 設定、靜默的 5 步上限與兩層重試

I'm Aman Kumar. Mastra is a TypeScript framework for agents and workflows, and most of its examples use model strings like "openai/gpt-4o" that resolve through a built-in provider registry. That's convenient until you want the same agent to talk to something that isn't in the registry: a self-hosted vLLM or LiteLLM server, Ollama's /v1 endpoint, or a hosted OpenAI-compatible gateway.

Mastra handles this with a small config object instead of a separate provider package. It works, but a few defaults surprised me, so I tested them with @mastra/core 1.78.0 on Node 20.19.2 against a local mock server that logs every request body and can return tool calls, 401, 429 with Retry-After, and 503 on demand. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1; I use YOUR_MODEL_ID below, so take real IDs from /v1/models or your provider's model list.

The minimal setup

npm i @mastra/core zod

import { Agent } from "@mastra/core/agent";

const agent = new Agent({
  id: "support",
  name: "support",
  instructions: "You answer in two sentences.",
  model: {
    id: "apiclaw/YOUR_MODEL_ID",
    url: "https://apiclaw.biz/v1",
    apiKey: process.env.APICLAW_API_KEY,
  },
});

const res = await agent.generate("What does HTTP 429 mean?");
console.log(res.text, res.usage);

The model field accepts a registry string, an AI SDK model instance, or this object. There is also a { providerId, modelId, url, apiKey } form. In my tests it produced exactly the same request.

Here is what the mock received:

POST /v1/chat/completions
{"model": "YOUR_MODEL_ID",
 "messages": [{"role": "system", "content": "You answer in two sentences."},
              {"role": "user", "content": "What does HTTP 429 mean?"}]}

The user agent was ai-sdk/openai-compatible, so under the hood this is the Vercel AI SDK's OpenAI-compatible provider. There was no GET /v1/models call and no extra parameters.

Four details about id and url

The part before the first slash is a label, not a lookup. apiclaw/ gets stripped and only the rest is sent as model. Only the first slash counts, so id: "apiclaw/deepseek/deepseek-chat" sends "model": "deepseek/deepseek-chat". That's what you want for gateways whose model IDs contain slashes, but it means you can't drop the prefix. Without url, Mastra tries the prefix against its registry, and an unknown one fails with No API URL found for apiclaw/YOUR_MODEL_ID.

url must include /v1. Mastra appends /chat/completions to whatever you pass, nothing more. With url: "http://127.0.0.1:8766" the request went to /chat/completions, which most servers answer with a 404. A trailing slash (.../v1/) was harmless: the request still went to /v1/chat/completions.

apiKey doesn't fall back to an environment variable. When I left it out and set APICLAW_API_KEY in the environment, the request went out with no Authorization header at all and the server returned 401. For a custom provider, read the variable yourself, as in the example above.

api: "responses" switches endpoints. With that flag the agent posted to /v1/responses with an input array instead of messages. Many OpenAI-compatible servers only implement Chat Completions, so leave it off unless your server documents the Responses API. My mock returned 404 and the agent threw AI_APICallError: Unknown path /v1/responses.

Custom headers work as you'd expect. headers: { "X-Team": "docs" } showed up on every request.

Tool loops stop at 5 steps, quietly

Tools are where I lost the most time. A normal tool call looked fine: the first request carried tools and "tool_choice": "auto", the model returned a tool call, Mastra ran my execute function and sent a second request with four messages (system, user, assistant tool call, tool result). That's two steps.

Then I had the mock keep requesting the tool forever, which is what a weaker model does when it can't decide it's finished:

import { createTool } from "@mastra/core/tools";
import { z } from "zod";

const weather = createTool({
  id: "get_weather",
  description: "Current weather for a city",
  inputSchema: z.object({ city: z.string() }),
  execute: async ({ city }) => ({ city, tempC: 31 }),
});

const agent = new Agent({ /* ...same model... */ tools: { weather } });
const res = await agent.generate("Weather in Pune?");
console.log(JSON.stringify(res.text), res.steps.length, res.finishReason);
// "" 5 tool-calls

The agent made five requests, ran the tool five times, and then returned normally. text was an empty string and finishReason was "tool-calls". It didn't throw an error or log a warning. If your code only reads res.text, the user just gets a blank answer.

Two fixes:

// raise or lower the budget per call
const res = await agent.generate(prompt, { maxSteps: 3 });

// and treat a run that ended on a tool call as unfinished
if (res.finishReason === "tool-calls" || !res.text) {
  // retry with a stronger model, or return a clear error
}

With maxSteps: 3 I got exactly three requests and three tool executions. Keep in mind that each step resends the whole conversation, so the fifth request carried ten messages. On per-token pricing that's where the cost goes. On a request-capped plan, one stuck question uses five requests.

Retries: there are two layers, and they multiply

This was the most surprising result. I pointed an agent at a model that always returned 503:

The first layer is the model-call retry from the AI SDK, set by modelSettings.maxRetries (or maxRetries on an entry in a model array). It's 0 by default in Mastra, it backs off exponentially (1, 2, 4, 8 and 16 seconds in my log), and it honoured Retry-After: 2 on a 429.

The second layer is Mastra's default error processors. They retry a failed generation twice, about three seconds apart, whatever maxRetries says. With both layers on, each of the three outer attempts ran its own six inner attempts, so one bad upstream turned into 18 requests and a 99-second wait. A 401 failed at once, with no retry.

What I settled on is one layer, set explicitly:

const agent = new Agent({
  // ...
  errorProcessorDefaults: false, // turn off the outer retry
});

await agent.generate(prompt, { modelSettings: { maxRetries: 2 } }); // 3 attempts, honours Retry-After

You can also keep the defaults and leave maxRetries at 0. Just don't turn up both, and if you have your own retry around generate(), that makes a third layer.

Fallbacks with the model array

model also accepts an array, and Mastra moves to the next entry when one fails:

model: [
  { model: { id: "apiclaw/YOUR_PRIMARY_MODEL", url, apiKey }, maxRetries: 1 },
  { model: { id: "apiclaw/YOUR_BACKUP_MODEL", url, apiKey }, maxRetries: 0 },
],

That worked, but with the default error processors still on, the primary got six requests (two inner attempts times three outer) before the backup got its first one, and the whole call took 9 seconds. If you rely on fallbacks, turn off the outer retry so the switch happens after one or two failures.

Streaming usage comes back empty

agent.stream() streamed text fine, but the request had no stream_options, so a server that follows OpenAI's behaviour sends no usage chunk, and await stream.usage came back with empty token counts. If you log tokens, build the model with the AI SDK provider directly and pass that instance to Mastra:

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const apiclaw = createOpenAICompatible({
  name: "apiclaw",
  baseURL: "https://apiclaw.biz/v1",
  apiKey: process.env.APICLAW_API_KEY,
  includeUsage: true,
});

const agent = new Agent({ id: "a", name: "a", instructions: "...", model: apiclaw("YOUR_MODEL_ID") });

With that, the request included "stream_options": {"include_usage": true} and usage reported input, output and total tokens. generate() reported usage either way.

Checklist

1. Use { id: "label/YOUR_MODEL_ID", url, apiKey }. Everything after the first slash is sent as the model name.
2. Include /v1 in url, and pass apiKey explicitly.
3. Leave api unset unless your server implements /v1/responses.
4. Check finishReason after tool runs, and set maxSteps on purpose.
5. Pick one retry layer: errorProcessorDefaults: false plus modelSettings.maxRetries, or the defaults with maxRetries at 0.
6. For token counts while streaming, use createOpenAICompatible with includeUsage: true.

Disclosure: I build APIClaw, the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.