Runcrate 團隊實測六種 GPU 從啟動到首次 CUDA 矩陣乘法的耗時

"How long until my GPU is ready?" is the question I care about most when I rent one for a short job, and it is the number nobody publishes. So this morning (2026-10-26) I launched single-GPU machines of six types on Runcrate through the API, one after another in one batch, and timed every step until a real CUDA matmul finished. Then I deleted each machine the second it was done.

What I measured

For each machine, seconds since the create call returned:

• API running: the instance API reports running and hands out the IP.
• First SSH: my script's first successful ssh root@ip echo ok (it retries every 2 s once it has the IP).
• First CUDA op: after SSH, install uv, create a venv, uv pip install torch --torch-backend=cu128, then multiply two 4096x4096 matrices on the GPU and synchronize.

I also read two clocks on the machine itself, because they tell you where the time goes: kernel boot (uptime -s) and when ssh.service came up (systemctl show ssh -p ActiveEnterTimestamp).

The commands

# 1. launch (instance_type_id from GET /api/v1/instances/types)
curl -s https://www.runcrate.ai/api/v1/instances \
  -H "Authorization: Bearer $RUNCRATE_API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"cold-l40s","instance_type_id":"<id>","ssh_key_id":"<key id>","max_runtime_minutes":30}'

# 2. poll every 10 s until status is running and ip is set
curl -s https://www.runcrate.ai/api/v1/instances/<id> -H "Authorization: Bearer $RUNCRATE_API_KEY"

# 3. on the machine
curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH=$HOME/.local/bin:$PATH
uv venv -q --python 3.12 /opt/env && . /opt/env/bin/activate
uv pip install -q torch --torch-backend=cu128
python -c "import torch,time;t=time.time();x=torch.randn(4096,4096,device='cuda');y=x@x;torch.cuda.synchronize();print(time.time()-t)"

# 4. delete
curl -s -X DELETE https://www.runcrate.ai/api/v1/instances/<id> -H "Authorization: Bearer $RUNCRATE_API_KEY"

Results

All VMs, Ubuntu image, one GPU each, launched within 4 seconds of each other at 09:02 UTC.

And from the machines' own clocks, same runs:

The torch step on its own (venv + wheel install + first matmul): 10 to 48 s for the install, 0.2 to 1.0 s for the first matmul, which is mostly CUDA context creation.

What surprised me

1. The spread is 4x, and it is not about the GPU. The L40S was doing math in about 3 minutes, the A6000 in almost 12. Same API, same image, same script. Cold start depends on where the machine comes from far more than on which card is in it, so if you launch often, measure the specific region you use.
2. On three types, the machine was ready long before the API said so. The V100, RTX 6000 Ada and A6000 booted and had sshd listening in 1 to 2 minutes, but the API reported them running about 8 minutes later. Nothing in my script could have used them earlier, because the IP arrives with running. The upside: billing starts at running, which is why the bills in the table are a cent or two.
3. The newest PyTorch wheels no longer run on a V100. uv pip install torch --torch-backend=cu128 resolved to torch 2.11.0, which warns that it was built for compute capability 7.5 and up, and the matmul then failed with cudaErrorNoKernelImageForDevice. The V100 is 7.0. On an earlier V100 run the same command had resolved to torch 2.8.0, which worked. If you rent Volta cards, pin torch (torch==2.8.* worked here) instead of taking the latest.
4. Not every create call works. Over the day I made 18 create calls. Two (an RTX 4090 and an RTX 5090 VM in the same region) came back within 7 seconds with 500 provider_unavailable, and one type was gone between listing it and launching it (400 ... no longer available). Both failures are fast and cost nothing, so the right client behaviour is: read /instances/types right before launching, and fall back to the next option on a 4xx/5xx.
5. A container is a different animal. In an earlier batch I launched an RTX 5090 as a container instead of a VM: its clock said it started 4 seconds after the create call. If startup time matters more than a full VM, check the deployment_type column.
6. My own polling made my first batch useless. I first polled nine machines every 2 seconds, hit the API's rate limit (HTTP 429) for several minutes, and the "running" times from that batch were wrong by minutes. This table is from a second batch polling every 10 seconds. Poll slowly.

Caveats

• One launch per GPU type in this table, at one time of day. Cold start depends on what is in stock at that moment; treat it as a sample, not a promise.
• "First CUDA op" includes downloading a 900 MB torch wheel. A pre-built image or a volume with a cached venv removes most of the 10 to 48 s.
• The prices are the on-demand prices at launch time and move daily.

Reproduce it

1. Create an API key, add an SSH key (POST /api/v1/ssh-keys).
2. For each type you care about: launch with the curl above, poll every 10 s or slower, SSH in as soon as there is an IP, run the torch block, delete.
3. Read uptime -s and systemctl show ssh -p ActiveEnterTimestamp on the machine if you want to know where the time went.

The six machines in the table were billed $0.07 in total.

If you want to time it yourself: runcrate.ai

Written by the Runcrate team, with AI assistance. The numbers come from the runs described above; raw logs are kept with the post.