48 小時實測筆記：我測試冷端點，卻把冷啟動延遲當成運算耗時

Why did the median look fine while the tail looked like a different machine?

I spent the first two days of a small measurement project convinced I had a slow code path. The median request landed in a tight cluster, the tail sat an order of magnitude above it, and the tail moved depending on which hour I sampled. So what happens when you average those two clusters into one number? You get a statistic that describes neither of them, and then you optimize the wrong thing.

My first ledger recorded exactly one value per request: total wall time from "start" to "response fully read". That single number mixed four phases together — name resolution, TCP connect, TLS handshake, and the server actually producing bytes. Cold infrastructure inflates the first three, while my own code inflates the fourth. Separating them took about forty lines, and it changed the entire conclusion.

What does a single duration number actually hide?

Before reaching for a profiler, ask which phase you are measuring, because a profiler will not answer a network question. Here is the split I started recording for every attempt:

1. connect — socket.create_connection returning.
2. tls — the handshake finishing on top of that socket.
3. send — the last request byte leaving my machine.
4. ttfb — the first response byte arriving.
5. total — the last byte arriving, plus the body size.

If ttfb dominates while connect stays flat, the endpoint is slow to think. If connect and tls jump together while ttfb stays flat, nothing is slow — something was asleep, and you measured the wake-up. Those two diagnoses lead to completely different fixes, which is exactly why the aggregate number was worthless to me.

The quick triage: one curl call, five timestamps

Before writing any Python, I confirmed the split with a single command. curl already records the phases, and this format string is the fastest way to see whether your tail is network or compute:

curl -sS -o /tmp/body.bin \
  -w 'dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total} bytes=%{size_download}\n' \
  https://your-endpoint.example/health

Run it twice in a row and compare the two lines instead of reading them once. A large gap between the first and second invocation is the clearest cold-start signal a shell can give you, and it costs two seconds to check. Remember that time_appconnect is zero on plain HTTP, so do not read that zero as "TLS was instant".

A probe that keeps the phases apart

Here is the artifact I ended up running in a loop. It is standard library only, it writes one JSON object per attempt, and it refuses to collapse the result into one duration.

# probe.py — phase-separated latency ledger
import json, socket, ssl, sys, time
from urllib.parse import urlparse

def probe(url: str, timeout: float = 15.0) -> dict:
    u = urlparse(url)
    host = u.hostname
    port = u.port or (443 if u.scheme == "https" else 80)
    path = u.path or "/"
    rec = {"ts": time.time(), "url": url, "ok": False}

    t0 = time.perf_counter()
    try:
        raw = socket.create_connection((host, port), timeout=timeout)
    except OSError as e:
        rec["error"] = f"connect:{type(e).__name__}"
        rec["total_s"] = round(time.perf_counter() - t0, 4)
        return rec
    rec["connect_s"] = round(time.perf_counter() - t0, 4)

    sock = (ssl.create_default_context().wrap_socket(raw, server_hostname=host)
            if u.scheme == "https" else raw)
    rec["tls_s"] = round(time.perf_counter() - t0, 4)

    sock.sendall((
        f"GET {path} HTTP/1.1\r\nHost: {host}\r\n"
        "User-Agent: coldstart-probe/1\r\nAccept: */*\r\n"
        "Connection: close\r\n\r\n"
    ).encode("ascii"))

    first = sock.recv(1)
    rec["ttfb_s"] = round(time.perf_counter() - t0, 4)
    rec["first_byte"] = first.decode("latin1") if first else None

    body = 0
    while True:
        chunk = sock.recv(65536)
        if not chunk:
            break
        body += len(chunk)
    sock.close()
    rec["body_bytes_after_first"] = body
    rec["total_s"] = round(time.perf_counter() - t0, 4)
    rec["ok"] = True
    return rec

if __name__ == "__main__":
    with open(sys.argv[2], "a", buffering=1) as fh:
        fh.write(json.dumps(probe(sys.argv[1])) + "\n")

Invoke it as python probe.py https://your-endpoint.example/health ledger.jsonl, then watch the file grow with tail -f ledger.jsonl. The buffering=1 line matters more than it looks: a killed loop keeps every observation it already made, which is a lesson I learned after an interrupted process left me with an empty file.

What broke during the first six hours

• My probe became part of the load. Polling once per second added a request per second, and the endpoint behaved differently under that traffic.
• Retries polluted the ledger. A retried request is not the same sample as a first attempt, and I had not tagged which was which.
• The loop had no stop condition. When the endpoint stopped answering, I collected hours of identical failures without noticing.
• A 200 in the log proved nothing about the payload. Body byte count, not status code, told me whether a response was complete.

Is a warm gate better than a bigger sample?

Yes, when your question is about steady-state behaviour, because a gate removes variance instead of averaging over it. Before timing anything, I now run this gate and record how many attempts it consumed:

def warm_gate(url, threshold_s=1.0, attempts=8, pause_s=2.0):
    for i in range(attempts):
        r = probe(url)
        if r.get("ok") and r["ttfb_s"] <= threshold_s:
            return i, r          # i == 0 means it was already warm
        time.sleep(pause_s)
    raise SystemExit(f"no warm response in {attempts} attempts")

Report that attempt count next to your timing results, because it is a result too. If the gate needs six attempts on every single run, the endpoint is re-initializing, and any steady-state number is valid only for the short window that follows the gate. A warm endpoint is not a fast endpoint; it is an endpoint whose cold phase is no longer inside your sample.

Where free model access and a free server fit into this workflow

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two parts of this loop are tedious rather than difficult, and that is where I leaned on MonkeyCode. Its free model access handled the glue work: generating probe variants for other runtimes, drafting the JSONL summarizer, and re-reading my curl format string every time I misread a zero. Its free server option let the 48-hour collection loop run somewhere other than my laptop, which mattered because my laptop sleeps and a sleeping collector produces gaps indistinguishable from an outage.

What it did not do is make the measurement trustworthy on its own. I am quoting no quotas, hardware, model names, or duration limits here, because I have not verified them and the claims above are operator-supplied availability claims. Treat any shared endpoint as a best-effort instrument, confirm its behaviour with the probe above, and keep the conclusion you draw proportionate to that uncertainty.

Decision table: is a shared endpoint the right instrument?

What I would repeat, and what I would change

1. Repeat: record connect, tls, ttfb, and total separately from the very first request.
2. Repeat: append to JSONL synchronously so a crash keeps the evidence.
3. Repeat: write down the stop condition before starting the loop, not after it ends.
4. Change: log whether each attempt followed an idle gap, since that column is what explained the split.
5. Change: run a paired comparison with alternating targets instead of two sequential campaigns.

Who should not use this approach

Skip it if your question concerns production capacity, hard real-time behaviour, or data under a regulated policy. Skip it if you need a number defensible in a contract, because a shared best-effort endpoint cannot supply one. And skip the long loop entirely when a two-minute paired run answers your question, since duration is not the same thing as rigour.

The takeaway I keep relearning

A benchmark is only as honest as its instrument, and an endpoint that sleeps at night is an instrument with a bad habit. The fix was not a faster machine or a larger sample; it was refusing to collapse four phases into one number and refusing to average two different populations together. Keep the ledger, draw the histogram before you compute the mean, and you will catch the cold start before it turns into a conclusion. If you run the probe and your tail splits in two, that histogram is worth far more than an average, so keep it around.