一個 API 對接多個模型供應商：供應商抽象化如何運作

One API can make many model providers look alike. Your app sends one request shape. A gateway then maps that request to OpenAI, Anthropic, Google, or another provider.

This can reduce code work. It does not make every provider the same.

Tool calls, streaming events, errors, model names, safety controls, and usage fields still differ. A strong abstraction keeps the common path simple and makes these gaps easy to see.

The Axiom Studio LLM Gateway offers one OpenAI-compatible endpoint across providers. This guide explains the design teams should test before they depend on any shared API.

Start with one stable request shape

Keep the app-facing request small. A basic request may include:

• A model or route name.
• System and user messages.
• Tool definitions.
• A choice between streaming and one final response.
• A request ID and team tag.

The gateway should validate this request before it reaches a provider. It should reject unknown fields and unsafe model choices with a clear error.

Do not hide the exact model that ran. Return the provider, model version, route decision, and usage data with the result. This helps people trace cost and behavior.

Map model names to routes

Apps should use a task name when possible, such as support-fast or code-review-deep. The gateway can map that name to an approved provider and model.

This keeps provider IDs out of app code. It also lets the platform team change a route without editing every service.

A route needs more than a model name. Store:

• Allowed teams and environments.
• Primary provider and fallback order.
• Timeouts and retry limits.
• Input and output caps.
• Data-region rules.
• Cost limits.

The unified AI gateway shows how routing, policy, cost controls, and audit records can share one control plane.

Tool calls need a strict contract

Tool calling is where many “compatible” APIs split.

One provider may stream a tool name first and arguments later. Another may return one complete tool block. One may allow several tool calls in the same turn. Another may use different rules for tool IDs or JSON fields.

Create one internal tool-call object with these fields:

• Tool name.
• Call ID.
• JSON arguments.
• Status.
• Provider data kept for debugging.

Validate the tool name and arguments before the app runs anything. Never repair broken JSON in silence. A repair can change what the model meant to do.

Google's official OpenAI compatibility guide says its compatibility support has limits. This is why teams must test the features they use instead of trusting the shared shape alone.

*Streaming needs one event model
*
Streaming feels simple when the output is only text. It gets harder when a stream includes reasoning notes, tool-call parts, safety stops, usage data, or provider errors.

Use a small internal event list:

• message_start
• text_delta
• tool_call_start
• tool_call_delta
• tool_call_end
• usage
• message_end
• error

Keep event order stable. Add sequence numbers. If a provider disconnects, tell the app whether the output is complete, partial, or safe to retry.

Do not join partial tool arguments and run them before the tool call is complete.

Normalize errors without losing details

Your app needs a short set of error groups:

• Bad request.
• Access denied.
• Rate limited.
• Provider busy.
• Timeout.
• Safety stop.
• Unknown provider failure.

Keep the provider status code and request ID in a protected log. Return a safe, useful error to the app.

Retry only errors that may improve on a second attempt. Do not retry a bad request. Use a retry cap and backoff. Count every retry in cost and latency.

Fallback is a product choice

A fallback can keep an app running. It can also change the answer.

Before fallback, check that the next model supports the same tools, output size, data rules, and response format. Mark the response when fallback occurs. Some work, such as a legal or health workflow, may need to stop instead of switching models.

The What Is an LLM Gateway? guide covers the request path, provider keys, logs, and controls that sit around routing.

Test the shared API with a contract suite

Run the same tests for every provider and model:

1. Plain text response.
2. Streaming text with a forced disconnect.
3. One valid tool call.
4. Bad tool arguments.
5. Two tool calls in one turn.
6. Rate-limit and timeout handling.
7. Maximum input and output behavior.
8. Usage and cost fields.
9. Safety stop behavior.
10. Fallback and retry limits.

Save the request, normalized events, final response, provider metadata, and result. Run the suite again when a provider changes an API version.

Measure the full cost

One endpoint can cut integration work. It can also add translation, retry, and support work.

Track:

• Cost per successful task.
• Retry and fallback rate.
• Time to first token.
• Full response time.
• Tool-call failure rate.
• Human review time.
• Provider-specific code that remains.

The AI FinOps guide explains how to connect model spend to teams and work. Use real usage fields from the provider response. Do not estimate cost from character count alone.

A safe rollout plan

Start with one low-risk task and two providers. Keep the old direct path ready for rollback.

Run both paths on the same saved requests. Compare output shape, tool behavior, errors, latency, and cost. Do not send customer data to a second provider unless your policy allows it.

Move one service at a time. Watch the contract tests and live error rate. Add a provider only when its gaps are known and owned.

One API should give teams one clear control point. It should not hide provider facts. The best abstraction keeps the common path easy and the differences visible.