小型 SaaS 自訂指標儀錶板：為何以回滾安全性比較 Push 與 Pull

Short answer: for a small SaaS tracking logistics notification failures, I would begin with a pull collector and a thin custom dashboard because a bad metric release can be rolled back with the application; I would choose a push API only when jobs are too short-lived or unreachable to scrape reliably. The dashboard is the last step. The important choice is which failure mode the team wants to own. This is a narrow decision, not a universal rule. The workload has delivery notifications, retry attempts, and final failures, and rollback safety is the primary axis. I care less about drawing the first chart than about proving that version N-1 can return without mixing incompatible metric names or labels from version N. That concern comes straight out of the notebook-to-production path: an exploratory counter feels harmless until an eval run, a retry worker, and a dashboard all disagree about what one event means.

What data flow keeps a delivery-failure dashboard understandable?

A notification worker receives a delivery event, attempts the configured channel, and updates an in-process counter using a deliberately small label set: channel, outcome, and metric schema version. A collector requests the worker's metrics endpoint on a schedule, stores the resulting time series, and the dashboard queries that store. With push, the worker sends the same observations to an ingestion endpoint instead. OpenTelemetry's metrics model is useful here because it separates the application-facing API from the SDK and export path; that boundary lets instrumentation remain stable while collection changes.

Keep the event definition boring. A final failure means the retry policy has been exhausted, not merely that the first SMS attempt timed out. notification_attempts_total and notification_final_failures_total answer different questions, so collapsing them creates an attractive but misleading failure-rate chart. I would also keep customer ID, shipment ID, phone number, and prompt text out of metric labels. They create open-ended dimensions and belong in logs or traces with appropriate controls, while metrics should carry bounded attributes suited to aggregation.

The schema version label is temporary deployment equipment. During a rollout, it makes mixed populations visible; after the old version is gone and the dashboard has been checked, recording rules or queries can remove that distinction. Don't use a version label as permanent decoration. It multiplies series and makes long-range panels harder to read.

Here is the smallest runnable shape I would put in front of the detailed trade-off. It uses only Python's standard library, exposes one conventional text endpoint, and snapshots counters under a lock so a scrape cannot observe a partially updated dictionary. The example data is intentionally bounded.

from collections import Counter
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Lock

COUNTERS = Counter()
COUNTER_LOCK = Lock()
ALLOWED_CHANNELS = {"email", "sms", "webhook"}
ALLOWED_OUTCOMES = {"delivered", "retry", "failed"}
SCHEMA_VERSION = "v2"


def record_notification(channel: str, outcome: str) -> None:
    if channel not in ALLOWED_CHANNELS:
        raise ValueError(f"unsupported channel: {channel}")
    if outcome not in ALLOWED_OUTCOMES:
        raise ValueError(f"unsupported outcome: {outcome}")

    with COUNTER_LOCK:
        COUNTERS[(channel, outcome, SCHEMA_VERSION)] += 1


def render_metrics() -> bytes:
    lines = [
        "# HELP notification_attempts_total Delivery notification attempts.",
        "# TYPE notification_attempts_total counter",
    ]
    with COUNTER_LOCK:
        snapshot = sorted(COUNTERS.items())

    for (channel, outcome, schema), value in snapshot:
        labels = (
            f'channel="{channel}",outcome="{outcome}",'
            f'schema="{schema}"'
        )
        lines.append(f"notification_attempts_total{{{labels}}} {value}")
    return ("\n".join(lines) + "\n").encode("utf-8")


class MetricsHandler(BaseHTTPRequestHandler):
    def do_GET(self) -> None:
        if self.path != "/metrics":
            self.send_error(404)
            return

        body = render_metrics()
        self.send_response(200)
        self.send_header("Content-Type", "text/plain; version=0.0.4")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, format: str, *args: object) -> None:
        return


if __name__ == "__main__":
    record_notification("sms", "delivered")
    record_notification("webhook", "retry")
    server = ThreadingHTTPServer(("127.0.0.1", 9108), MetricsHandler)
    server.serve_forever()

Run it, request 127.0.0.1:9108/metrics, and you have enough surface area to test collection without coupling business code to a dashboard. In production I would use maintained instrumentation rather than grow this sample into a home-made client library. The point is the boundary: record_notification knows the metric contract, while transport remains replaceable.

Small detail, big payoff.

How should a beginner compare a push API with a pull model?

Compare ownership of time, identity, and failure. In a pull model, the collector decides when to ask, records whether a target was reachable at that moment, and associates the result with its target configuration. In a push design, the application decides when to send, so it needs a destination, authentication, buffering policy, retry policy, timeout, and a way to avoid confusing a retried payload with a new observation. Those are architectural consequences, not a scorecard declaring one transport better.

For this delivery service, rollback behavior breaks the tie. Suppose release N changes outcome="error" to outcome="failed", then the eval harness finds that retry exhaustion is being classified incorrectly. With pull, rolling the workers back restores code and exposition together. The store may still contain both label values during the deployment window, but the collector has not accepted an opaque queue of future writes from the reverted version. With push, queued batches from N can arrive after N-1 is live again unless the export protocol, queue, and receiver preserve enough metadata to separate them. That design can be correct, but it is more policy to test.

I wouldn't hide that catch: a pull endpoint is not suitable when a batch process often finishes between collection intervals, when the collector cannot reach the workload, or when exposing a listener violates the deployment boundary. Choose push in those cases, then make queue age, retry limits, and drop behavior part of the contract. Conversely, stick with pull when long-running services are reachable and the team values one place to control collection cadence. Your mileage may vary in serverless environments because lifetimes and network paths differ; a deployment trace showing actual instance duration and connectivity resolves that uncertainty better than preference does.

The familiar tools illustrate distinct layers rather than a forced bundle. Prometheus is associated with pull collection and time-series queries, Grafana renders dashboards over data sources, and OpenTelemetry defines APIs, SDK responsibilities, and metric export concepts. A StatsD-style emitter is another push-shaped option for small numeric updates. These names clarify the architecture, but the decision should survive replacing every name in the diagram.

Rollback safety needs an eval, not a hopeful deploy

Before rollout, feed a fixed fixture of 1,000 synthetic notification results through both metric-schema versions. The fixture should include first-attempt delivery, retry-then-delivery, and exhausted retries across each allowed channel. Assert invariants rather than pixel positions: total attempts equal the sum by outcome, final failures never exceed attempts, every emitted label belongs to an allowlist, and the old dashboard query returns a result while both versions are running. The number 1,000 is a test-fixture size, not a production benchmark; use enough cases to cover the state machine deterministically.

Then rehearse the rollback. Start N-1 and N together, collect two intervals, stop N, and verify that the query neither disappears nor double-counts cumulative counters. Counter resets are expected when a process restarts, so calculate rates with a query engine that understands cumulative counters rather than subtracting two raw samples in dashboard code. OpenTelemetry documents monotonic sums and aggregation temporality, which is the standard vocabulary I use when reviewing this behavior.

I've learned to treat metric changes like prompt changes: both alter the observable contract, and both need a fixture with expected outputs. The cost angle is similar too. A label that accepts shipment IDs can create a new series per delivery, just as an unbounded prompt can consume tokens without improving the eval. I can't give a universal series budget because retention, scrape cadence, aggregation, and backend implementation all matter. Measure active series and ingestion volume during the canary, then set a budget from the system you actually operate.

No vibes.

The dashboard itself should begin with three views: attempt rate by channel, final-failure ratio by channel, and retry volume. Add a deployment annotation or equivalent release marker so an operator can compare the metric-schema transition with application changes. Avoid an enormous wall of panels. A beginner gets more rollback confidence from three tested queries than from twenty panels whose label assumptions are undocumented.

Operating the choice after launch

The operational checklist is prose because these checks form one loop. On every metric-contract change, run the fixture against old and new code, inspect the endpoint locally, and canary both versions. During the canary, watch target reachability or export drops, active series, collection latency, and the three delivery panels. Roll back if the invariants fail, even when the graphs look plausible. After the rollout settles, remove compatibility queries on a scheduled date and record the accepted schema in the repository beside the eval.

Also test the observability path itself. Block collection briefly in a staging environment and confirm that the team can distinguish "no notifications happened" from "metrics were not collected." For pull, collector-side target health supplies one part of that evidence. For push, instrument the exporter queue and dropped-record path independently of the application metric being exported. Don't let the notification failure counter be the only signal that its own transport works.

The final decision is conditional: use pull for the long-running, reachable notification workers because collection health and rollback boundaries stay easier to inspect; use push for ephemeral or unreachable jobs, with explicit durability and deduplication tests. Either can support a small SaaS custom metrics dashboard. The safer choice is the one whose failure semantics your team has actually rehearsed.

Sources

• https://opentelemetry.io/docs/concepts/signals/metrics/
• https://prometheus.io/docs/instrumenting/exposition_formats/
• https://prometheus.io/docs/practices/pushing/
• https://grafana.com/docs/grafana/latest/dashboards/