Amazon Bedrock 與 EKS 自行託管 LLM 的損益平衡成本分析

When do your own GPUs beat Amazon Bedrock for the same open-weight model? Official AWS prices, the tokens per second you need to break even, and the Karpenter and vLLM config to test it.

By Evgeny Anikiev, AWS-certified DevOps engineer with 10+ years running build automation, CI/CD, and Kubernetes in production. Prices pulled from the AWS Price List API on October 23, 2026 (US East, N. Virginia, on-demand list prices).

TL;DR: The same open-weight model costs per token on Bedrock and per hour on your own GPUs. One H100 on a p5.4xlarge running gpt-oss-120b costs $5,022 a month. To beat Bedrock's price for that model, it has to process about 7,300 tokens per second, around the clock, every day of the month. Unless your traffic is that high and that steady, Bedrock costs less. Self-hosting also makes sense when you need a model Bedrock doesn't offer.

The numbers in this post:

Bedrock prices for gpt-oss-120b, gpt-oss-20b, and Llama 3.3 70B.
EC2 GPU instance prices for the instances that fit each model.
The break-even volume in tokens per month and tokens per second.
The costs a per-token price hides and a per-hour price doesn't.
A Karpenter NodePool and vLLM Deployment to run your own benchmark.

Which costs less: Amazon Bedrock or self-hosting on EKS?

Unless a GPU stays busy around the clock, Bedrock costs less. Bedrock bills per token and costs nothing when idle. A self-hosted model on EKS bills per GPU-hour whether it serves traffic or not. Self-hosting only wins when a GPU stays busy enough that its hourly cost divided by its tokens drops below Bedrock's per-token price. For gpt-oss-120b on one H100, that point is about 19 billion tokens a month.

Bedrock and self-hosting on EKS, side by side:

Billing: Bedrock charges per million tokens. EKS charges per GPU-hour, plus $73 a month for the cluster control plane.
Idle cost: zero on Bedrock. Full hourly price on EKS until the node is removed.
Scaling: Bedrock scales within your tokens-per-minute quota. On EKS, you scale GPU nodes yourself, with cold starts of several minutes.
Models: Bedrock offers its catalog plus Custom Model Import. EKS runs any open-weight model that fits your GPUs.
Operations: AWS runs Bedrock. On EKS, your team owns drivers, the serving engine, autoscaling, and on-call.

What does each option cost?

Bedrock: per token

These are on-demand (Standard tier) prices per million tokens in US East, from the AWS Price List API:

gpt-oss-120b: $0.15 input, $0.60 output.
gpt-oss-20b: $0.07 input, $0.30 output.
Llama 3.3 70B Instruct: $0.72 input, $0.72 output.

Batch inference and the Flex tier halve these prices for work that can wait, for example $0.075 input and $0.30 output for gpt-oss-120b.

Agents send far more input than they get back, because every call resends the conversation and tool results. With 3 input tokens for every output token, the blended price per million tokens is:

gpt-oss-120b: $0.2625
gpt-oss-20b: $0.1275
Llama 3.3 70B: $0.72

Self-hosted: per GPU-hour

On-demand Linux prices in US East:

g6e.xlarge (1× NVIDIA L40S, 48 GB): $1.861 per hour, about $1,359 a month.
p5.4xlarge (1× NVIDIA H100, 80 GB): $6.88 per hour, about $5,022 a month.
g6e.12xlarge (4× L40S, 192 GB total): $10.49 per hour, about $7,660 a month.

Which instance fits which model:

gpt-oss-20b fits on one L40S (g6e.xlarge). OpenAI says it runs within 16 GB of memory.
gpt-oss-120b fits on one 80 GB GPU, per OpenAI, so one H100 (p5.4xlarge) works. Four L40S GPUs (g6e.12xlarge) also fit it, at a higher hourly price.
Llama 3.3 70B needs about 140 GB for its weights at 16-bit precision, so it needs g6e.12xlarge or larger.

Where is the break-even point?

Divide the monthly instance cost by Bedrock's blended price. The result is how many tokens the instance has to process each month to cost the same as Bedrock. Divide by the seconds in a month to get the tokens per second it has to sustain.

HOURS = 730  # hours in an average month

def break_even(instance_per_hour, bedrock_in, bedrock_out, ratio=3):
    """Prices: instance in USD/hour, Bedrock in USD per million tokens.
    ratio = input tokens per output token."""
    blended = (ratio * bedrock_in + bedrock_out) / (ratio + 1)
    monthly = instance_per_hour * HOURS
    tokens = monthly / blended * 1_000_000
    per_second = tokens / (HOURS * 3600)
    return round(monthly), round(tokens / 1e9, 1), round(per_second)

print(break_even(6.88, 0.15, 0.60))      # gpt-oss-120b on p5.4xlarge:   (5022, 19.1, 7280)
print(break_even(1.861, 0.07, 0.30))     # gpt-oss-20b on g6e.xlarge:    (1359, 10.7, 4054)
print(break_even(10.49264, 0.72, 0.72))  # Llama 3.3 70B on g6e.12xlarge: (7660, 10.6, 4048)

Each instance has to sustain this load 24 hours a day to match Bedrock:

gpt-oss-120b on one H100: 19.1 billion tokens a month, about 7,300 tokens per second.
gpt-oss-20b on one L40S: 10.7 billion tokens a month, about 4,100 tokens per second.
Llama 3.3 70B on four L40S: 10.6 billion tokens a month, about 4,000 tokens per second.

Two adjustments make the bar higher:

Real traffic isn't flat. If your GPU is busy 40% of the time on average, it needs to handle 2.5 times the break-even rate at peak. For gpt-oss-120b, that is about 18,000 tokens per second.
If your workload can wait, compare against Batch or Flex. Both are half price, which doubles every break-even number above.

Whether one GPU can hit these rates depends on your prompts, context length, batch size, and serving engine. Measure it. The config at the end of this post sets up that test.

Worked example: the support agent from the readiness checklist

This example is hypothetical. The support agent from my AI agents readiness checklist handles 10,000 tasks a month, with 6 model calls per task, 8,000 input tokens and 1,000 output tokens per call:

Input: 480 million tokens a month.
Output: 60 million tokens a month.

On Bedrock:

gpt-oss-120b: 480 × $0.15 + 60 × $0.60 = $108 a month.
gpt-oss-20b: 480 × $0.07 + 60 × $0.30 = $51.60 a month.

Self-hosting the same agent on one p5.4xlarge costs $5,022 a month, about 46 times more. Its 540 million tokens are under 3% of the 19.1 billion the H100 needs to break even, so the agent would need about 35 times its current volume before that GPU paid for itself.

What costs does the per-hour price leave out?

Add these to the GPU-hour:

The EKS control plane: $0.10 per hour, $73 a month per cluster, if you don't already run one.
Model downloads. gpt-oss-120b weights are tens of gigabytes. Every new node pulls them. If that goes through a NAT gateway from S3 or the internet, you pay NAT processing on every pull. An S3 gateway endpoint makes the S3 part free; see Cut AWS NAT Gateway Costs.
Cold starts. Scaling GPU nodes to zero saves money but means minutes of waiting for a node, an image, and the weights before the first answer. Keeping one node warm puts you back on the 24-hour cost.
Quotas. EC2 limits GPU instances through vCPU quotas for G and P instances. Check them before you plan on p5 instances, since an increase request can take days.
People. Someone owns the serving engine upgrades, GPU drivers, autoscaling, and on-call. Bedrock includes that in the token price.

Savings Plans and EC2 Capacity Blocks lower the hourly price for steady GPU use, which moves the break-even point down. Price them for your term and Region before deciding.

When does self-hosting on EKS make sense?

Self-host when at least one of these is true:

Steady, high volume. Your measured traffic stays above the break-even rate for most of the day.
A model Bedrock doesn't have. A fine-tuned or niche open-weight model that isn't on Bedrock and can't be imported.
Control over the serving stack. You need a specific engine, quantization, or batching setup that a managed API doesn't expose.

Stay on Bedrock when:

Traffic is spiky or still growing. Paying per token means idle hours cost nothing.
You switch models often. Bedrock lets you change the model ID instead of resizing GPU node pools.
Nobody on the team wants GPU on-call.

A practical path is to prototype on Bedrock, measure real token volume for a month, then run the break-even function above with real numbers.

How do I test self-hosting on EKS?

Two pieces: a Karpenter NodePool that launches GPU nodes only when a GPU pod is waiting, and a vLLM Deployment that serves the model with an OpenAI-compatible API.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-inference
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["g6e"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
      taints:
        - key: nvidia.com/gpu
          effect: NoSchedule
  limits:
    nvidia.com/gpu: 4
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 10m

The taint keeps other pods off GPU nodes. WhenEmpty removes a GPU node 10 minutes after its last pod leaves, so a test cluster doesn't bill GPU hours overnight. The limits block caps the pool at four GPUs, which stops a runaway Deployment from launching more.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gpt-oss-20b
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gpt-oss-20b
  template:
    metadata:
      labels:
        app: gpt-oss-20b
    spec:
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: vllm
          image: vllm/vllm-openai:latest
          args: ["--model", "openai/gpt-oss-20b"]
          ports:
            - containerPort: 8000
          resources:
            limits:
              nvidia.com/gpu: 1

Pin the vLLM image to a specific version once your test works. Then replay a day of real prompts against both endpoints, measure sustained tokens per second at your latency target, and put that number next to the break-even rate.

FAQ

Is Amazon Bedrock cheaper than running an LLM on EC2?

Unless the GPU stays busy around the clock, yes. Bedrock charges only for tokens processed, while an EC2 GPU instance charges every hour it runs. A single H100 serving gpt-oss-120b has to sustain about 7,300 tokens per second, 24 hours a day, to match Bedrock's on-demand price for that model.

How much does it cost to run gpt-oss-120b on AWS?

On Bedrock, $0.15 per million input tokens and $0.60 per million output tokens on demand in US East, or half that with Batch or Flex. Self-hosted on one p5.4xlarge (one H100), $6.88 per hour, about $5,022 a month on demand.

What EC2 instance do I need for gpt-oss-20b?

One GPU with at least 16 GB of memory. A g6e.xlarge with one NVIDIA L40S (48 GB) costs $1.861 per hour on demand in US East and leaves room for long contexts and batching.

Can I scale GPU nodes to zero on EKS?

Yes. Karpenter removes empty nodes based on the NodePool's disruption policy and launches new ones when a GPU pod is pending. The trade-off is cold start time, since a new node has to boot, pull the container image, and load the model weights before serving.

Is SageMaker cheaper than EKS for self-hosting an LLM?

The break-even logic is the same. A SageMaker real-time endpoint also bills per instance-hour while it runs, so it has to stay as busy as an EKS GPU node to beat Bedrock's per-token price. Compare the instance-hour prices for your Region, then run the same calculation.

Does Bedrock have a cheaper option for steady high volume?

Yes. Bedrock's Reserved service tier commits to a fixed tokens-per-minute capacity, with a minimum of 100,000 input and 10,000 output tokens per minute, arranged through your AWS account team. If your volume is close to the self-hosting break-even point, get a Reserved quote before buying GPUs.

Does Bedrock offer the same open-weight models I can self-host?

Some of them. gpt-oss-120b, gpt-oss-20b, and Llama 3.3 70B are available on Bedrock as serverless models. For a model Bedrock doesn't list, check Bedrock Custom Model Import before deciding to self-host.

Sources

Deciding between Bedrock and your own GPUs? I run cost reviews and build EKS platforms for AI teams at DevOps Cloud Consult. Bring a month of token counts and I'll run the break-even for your workload.

Get one AWS cost fix every two weeks: subscribe on Substack.