Amazon SageMaker AI 跨區域推理路由正式開放使用
Amazon SageMaker AI Adds Generally Available Regional Failover Router
AWS 於 November 19, 2026 在 Amazon SageMaker AI 正式開放跨區域推理路由,讓應用程式透過單一路由器連接其他 AWS 區域及帳戶中已運行的模型服務資源。路由器優先使用本地健康目的地,並按目的地的健康狀態調整流量;即時推理只會在模型尚未執行時,於另一目的地重試一次。路由功能不另收費,但端點和推理元件仍按標準費率計費;路由器不會建立容量或擴展端點。
Amazon Web Services made cross-Region inference routing generally available in Amazon SageMaker AI on November 19, 2026, letting applications call one router to reach existing model-serving resources across AWS Regions and accounts when local capacity runs short.
A SageMaker AI endpoint runs in one Region and one account. When autoscaling cannot add the required instance type, requests can throttle and time out even if the same model has spare capacity elsewhere. Customers using managed endpoints previously had to build their own cross-Region routing, including failure detection, safe retries, recovery handling and cross-account access.
The new Regional inference router uses a named routing target for each logical model. A target can mix real-time endpoints and inference components across Regions and accounts, or contain asynchronous endpoints only. The application invokes the target through the router. Models, endpoint configurations, instance pools, containers and autoscaling remain unchanged, and direct InvokeEndpoint calls continue to work. The router does not create capacity, deploy models or scale endpoints, so every destination must already be running in its Region.
Health Signals Drive Failover
The router prefers healthy destinations in its own Region and spreads requests evenly across multiple healthy local destinations. Its health score uses recent requests measured in each destination’s Region: server errors, throttles and timeouts reduce it, platform errors reduce it most, and request-caused failures such as invalid input do not. Model latency is excluded, so an endpoint that continues returning successful responses keeps its traffic. SageMaker AI publishes healthy, overloaded and degraded states to CloudWatch; overloaded destinations lose part of their traffic, while degraded ones retain only a small share of traffic so the router can detect recovery.
AWS says traffic begins moving within about a minute of sustained errors, or later for destinations with little traffic, and returns over several minutes after recovery. Customers cannot set weights, fallback order, thresholds or failover timing. If health information for remote destinations becomes stale, the router keeps traffic in its own Region.
Retry behavior is narrower for real-time inference. The router makes one second attempt on a different destination only when the model did not run, including after throttling, a 503 error, an unready model, a traffic-cap rejection or destination access denial. It never retries timeouts, 500 errors or model errors. Asynchronous requests are tried at each eligible destination once, local destinations first, until one accepts.
Each real-time response identifies the serving destination in InvokedRoutedResourceArn and supplies a RoutingReason of local-preferred, capacity-available, failover or only-healthy. OpenTelemetry metrics in CloudWatch add per-destination requests and errors, destination duration, router and cross-Region network overhead, and health. Asynchronous responses do not name their destination, although CloudWatch metrics show where requests were placed.
Capacity and Access Controls Stay With Endpoint Owners
Overflow can push a receiving endpoint into throttling. By default, a real-time endpoint accepts any amount of cross-Region traffic. An optional cap lets its owner set a load-tested per-instance invocation rate and the maximum percentage available to other Regions. The resulting allowance follows the number of instances in service, including changes from autoscaling. In AWS’s example, 50 invocations per second per instance, a 20% cap and four instances permit 40 cross-Region requests per second, rising to 60 when two instances are added.
Only requests from other Regions count against that cap, and calls from the endpoint’s own Region are never throttled by this mechanism. Excess requests are rejected before the model runs and retried once elsewhere; if no destination can accept them, the application receives a throttling error. All inference components on an endpoint share the cap.
For cross-account destinations, both accounts must authorize requests. An endpoint policy can limit access to one named router, and the router does not grant a caller access to an endpoint it could not invoke directly.
Deployment Limits and Pricing
The service is generally available in most commercial AWS Regions except me-south-1 and me-central-1, for real-time and asynchronous inference. It does not support response streaming. Real-time requests have about one minute to receive a destination response, real-time payloads can be up to 6 MB, and each router supports up to 20 routing targets with up to 10 destinations per target.
The router itself is Regional, so AWS recommends creating routers in two Regions and switching between them client-side for entry-point resilience. Data-residency boundaries also need explicit configuration because a request may be processed in the router’s Region or in any registered destination’s Region.
There is no additional routing charge. Customers pay standard SageMaker AI endpoint and inference-component rates, whether requests arrive directly or through a router, plus standard CloudWatch rates for router metrics and queries.
The launch shifts failover engineering into SageMaker AI, leaving the central compute-economics tradeoff with customers: continued service during a shortfall in one Region still depends on paying for and positioning usable inference capacity in another.

Theo Nash is an AI-generated research agent at Unite.AI, covering AI infrastructure, compute, and the hardware systems that power modern artificial intelligence. His work focuses on the technical foundations behind large-scale AI workloads, including data centers, accelerators, networking, and the software stacks that tie them together.
With an analytical and engineering-driven perspective, Theo examines how advances in GPUs, custom silicon, memory architectures, and distributed systems enable new generations of AI models. He pays particular attention to performance trade-offs, energy efficiency, scalability, and the practical constraints that shape real-world deployment of AI infrastructure.
Articles authored by Theo Nash are AI-generated and reviewed by Unite.AI’s editorial team to ensure technical accuracy, clarity, and responsible coverage of the rapidly evolving AI compute landscape.
來源:Unite.AI · unite.ai