健康檢查顯示正常，伺服器閒置後的首個請求仍可能變慢

You pinged /health. It said 200. Then the page took four seconds.

You check a health endpoint before a demo. It answers in 40 ms. Then you load the real page.

Four seconds of nothing. Then everything works, until you go idle again.

So which request did you actually measure? Not the one your users make.

Why a free box is even in this conversation

I run small probes and demo apps on a free server option. I also run throwaway evaluations against free model access.

That is exactly where cold-start confusion bites. You get real infrastructure. You do not get a warm one.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two rules keep me honest:

1. Measure the first request after idle, not the steady state.
2. Treat every free box as disposable until a probe says otherwise.

Here are the myths I hear most, and what actually holds up.

Myth 1: "200 OK means the app is warm"

The claim

If /health returns 200, the service is ready for traffic.

The evidence against it

Readiness and warmth are different properties:

• Readiness answers: is the process accepting connections?
• Warmth answers: how long does a real request take right now?
• A trivially cheap endpoint can be ready while everything expensive is cold.

Your /health handler may not touch the template engine, the DB pool, or the auth cache. The real route touches all three.

Corrected model

Warmth is a latency distribution, not a boolean. You do not "have" it. You measure it per request.

Myth 2: "Idle time is free time"

The claim

An unused instance costs nothing and behaves the same when you return.

The evidence against it

The bill arrives on the first request after idle. That request pays for process start, imports, and socket setup.

If you average latency over a day of traffic, the cold request disappears. Your users never see averages. They see the first one.

Corrected model

Bucket your latency by idle gap. A p50 over all requests hides the only number you care about.

Ask yourself: when did I last measure a request after 30 minutes of silence?

Myth 3: "The filesystem remembers"

The claim

You wrote a file yesterday, so it is there today.

The evidence against it

Many free hosting options give you an ephemeral filesystem. Anything outside your deployed artifact can vanish on restart, redeploy, or scale event.

Test it with a sentinel file. If it disappears, that was never storage. That was a cache with extra steps.

Corrected model

State belongs in an external store or in your deploy artifact. The box holds neither.

Myth 4: "uptime tells you how long your app has run"

The claim

uptime printed 12 days, so the app has been running for 12 days.

The evidence against it

In a container, /proc/uptime reports the host kernel's uptime. It does not report your container's age.

Your process may have restarted five times under a host that never rebooted. The clock stays calm. Your app does not.

Corrected model

Use three separate signals:

1. boot_id from /proc/sys/kernel/random/boot_id, to detect host or VM resets.
2. A sentinel file with an mtime, to detect filesystem resets.
3. Your app's own start timestamp, written at import time.

Myth 5: "A retry fixes it"

The claim

If the first call is slow or fails, just try again.

The evidence against it

Retries help with a cold start. They do not help with lost state. Retrying a stateful call after a filesystem reset can double-apply work.

And on a model endpoint, a blind retry can double your token spend on a request that already succeeded slowly.

Corrected model

Retry only idempotent reads. For writes, check the sentinel first. For model calls, log the outcome before you decide to resend.

The probe: measure warmth, not liveness

The script below uses only the Python standard library. It records one JSON line per run.

#!/usr/bin/env python3
# warmth_probe.py - cold start vs steady state, stdlib only
import json, os, sys, time, uuid, urllib.request
from datetime import datetime, timezone

SENTINEL = os.environ.get('SENTINEL', '/tmp/warmth_sentinel')
URL = sys.argv[1] if len(sys.argv) > 1 else 'http://127.0.0.1:8080/health'

def read(path, cast=str):
    try:
        with open(path) as f:
            return cast(f.read().split()[0])
    except (OSError, ValueError):
        return None

def probe():
    started = time.monotonic()
    try:
        with urllib.request.urlopen(URL, timeout=30) as r:
            status, body = r.status, r.read(64)
    except Exception as e:
        status, body = -1, str(e).encode()
    elapsed_ms = round((time.monotonic() - started) * 1000, 1)

    sentinel_age = None
    if os.path.exists(SENTINEL):
        sentinel_age = round(time.time() - os.path.getmtime(SENTINEL), 1)
    else:
        with open(SENTINEL, 'w') as f:
            f.write(uuid.uuid4().hex)

    return {
        'ts': datetime.now(timezone.utc).isoformat(),
        'status': status,
        'elapsed_ms': elapsed_ms,
        'boot_id': read('/proc/sys/kernel/random/boot_id') or 'unknown',
        'host_uptime_s': read('/proc/uptime', float),
        'sentinel_age_s': sentinel_age,   # null means this filesystem is new
        'body_prefix': body[:32].decode('utf-8', 'replace'),
    }

print(json.dumps(probe()))

Run it, append, repeat:

chmod +x warmth_probe.py
./warmth_probe.py 'http://127.0.0.1:8080/health' >> warmth.jsonl

Schedule it unevenly on purpose

A fixed 5-minute cron never sees long idle. You will measure warm requests forever and learn nothing.

Use two schedules instead:

# frequent sampling
*/5 * * * * cd /app && python3 warmth_probe.py 'http://127.0.0.1:8080/health' >> warmth.jsonl

Then run one manual probe after a long gap, such as the first request of your morning. That single line is the interesting one.

Reading the log

import json
from datetime import datetime

rows = [json.loads(line) for line in open('warmth.jsonl')]
rows.sort(key=lambda r: r['ts'])

def pct(xs, p):
    xs = sorted(xs)
    return xs[max(0, min(len(xs) - 1, round(p / 100 * (len(xs) - 1))))]

fresh = [r['elapsed_ms'] for r in rows if r['sentinel_age_s'] is None]
warm = [r['elapsed_ms'] for r in rows if r['sentinel_age_s'] is not None]

print('filesystem resets:', len(fresh))
print('boot_id values:', len({r['boot_id'] for r in rows}))
print('warm p50:', pct(warm, 50), 'warm p95:', pct(warm, 95))

buckets, prev = {}, None
for r in rows:
    t = datetime.fromisoformat(r['ts'])
    if prev:
        gap = (t - prev).total_seconds()
        key = 'under 1m' if gap < 60 else 'under 15m' if gap < 900 else '15m plus'
        buckets.setdefault(key, []).append(r['elapsed_ms'])
    prev = t

for key, xs in sorted(buckets.items()):
    print(key, 'n =', len(xs), 'p50 =', pct(xs, 50))

Example output shape, not a benchmark:

filesystem resets: 3
boot_id values: 2
warm p50: 41.0 warm p95: 88.2
under 1m n = 96 p50 = 44.0
15m plus n = 5 p50 = 2140.5

If the last line looks like that, your health check was never lying. It was answering a different question.

A decision table

Limitations, and who should skip this

This probe has honest limits:

• It measures from inside the box. Platform queueing before your process starts is invisible.
• /proc is Linux-only. On other systems, check boot_id and fall back to your own start timestamp.
• A single sentinel file cannot detect partial resets in subdirectories.
• Free offerings change. A probe log from last month proves last month.
• The probe itself sends traffic. Do not loop it against a rate-limited endpoint.

Skip this approach if you need contractual uptime, handle regulated data, or run a stateful service. A free box is a lab bench, not a stage.

What to do in the next ten minutes

1. Drop the probe next to your app and point it at the real route, not /health.
2. Run it once. Then wait. Run it again after you go get coffee.
3. Compare the two lines. Bucket by gap, never by average.
4. Delete every piece of state that is not in your deploy artifact.

The health check was green the whole time. The question is whether your measurement was.

If you try this against a free model endpoint as well, log the outcome of each call before you log the latency. That is where the surprises hide. Running the probe on a free server option is a cheap way to see your own idle policy instead of guessing it.