如何在 Node.js 建立四步功能旗標服務，追蹤智能體切換成本

TL;DR: keep the toggle service small, but make every decision explainable. Store versioned flag rules, evaluate them deterministically, and emit one structured decision event with the agent run ID, tenant, flag key, result, latency, and rule version. That event is the join point between a feature rollout and the cost of an AI agent loop.

The example here uses the middle option: a tiny service with a read endpoint and an explicit evaluation record. The application is an education assistant that may call retrieval, a model, and a human-review queue. Four decisions can change one loop: which prompt template is active, whether retrieval is enabled, which model tier is allowed, and whether a response needs review. Cost attribution is the decision axis, so the flag result must travel with the run rather than disappear in an application log.

That is the contract.

Measure it.

What should a toggle decision prove?

A boolean is not enough. An operator needs to answer: which rule selected this value, for which tenant, at what time, and under which configuration version? The service should return a value plus metadata. The caller then attaches that metadata to the trace or log record for the agent run.

A useful mental diagram is: request -> flag evaluation -> agent span -> model and tool spans -> decision event. The arrows matter. If evaluation happens outside the trace, a later cost report can show an expensive model call but cannot explain why that branch was selected.

Use stable identifiers. A flag key such as agent.model_tier is easier to aggregate than a sentence. A tenant identifier should be pseudonymous where possible. Do not put learner prompts, answers, or access tokens into the decision event.

How can you build a feature flags service in Node.js with deterministic toggle decisions?

Keep the evaluation function boring. It accepts a snapshot and a context, checks an exact tenant override before the default, and returns the rule version used. The snapshot can be held in memory by the HTTP service and replaced atomically after an administrative update.

type FlagContext = {
  tenantId: string;
  runId: string;
};

type FlagRule = {
  key: string;
  defaultValue: boolean;
  tenantValues: Record<string, boolean>;
  version: number;
};

type FlagDecision = {
  key: string;
  value: boolean;
  version: number;
  source: "tenant" | "default";
};

export function evaluateFlag(
  rule: FlagRule,
  context: FlagContext,
): FlagDecision {
  const override = rule.tenantValues[context.tenantId];

  if (override !== undefined) {
    return {
      key: rule.key,
      value: override,
      version: rule.version,
      source: "tenant",
    };
  }

  return {
    key: rule.key,
    value: rule.defaultValue,
    version: rule.version,
    source: "default",
  };
}

The endpoint can expose this function behind a normal Node.js HTTP handler. Validate the flag key, require a tenant context, and reject an unknown key rather than silently returning false. A silent false is particularly costly here: it can switch an agent to a different retrieval or model path while making the resulting spend look like ordinary traffic. In a small deployment, a 60-second request timeout and a 15-minute refresh-age alert are reasonable starting points; tune them against the agent's latency budget and rollout risk.

Emit the event after the decision is made. Include runId, tenantId, flagKey, value, source, version, and an evaluation duration in milliseconds. Record the same runId on the agent span and model-call spans. OpenTelemetry treats logs as a signal that can carry structured attributes, and its signal model gives teams a common vocabulary for correlating telemetry; the exact exporter is an implementation choice.

One practical detail: never measure only the model call. Measure flag lookup separately. A remote toggle lookup adds network latency, while an in-process snapshot shifts the problem toward refresh consistency. Put both measurements on the run timeline so a rollout does not get blamed for time spent in retrieval.

Which option fits the rollout boundary?

In-process defaults fit a single worker with a deploy-driven release process. They are easy to reason about, but each process can carry a different configuration during a rolling deployment. A small HTTP service fits shared tenant rules and makes the source of truth explicit; it needs timeouts, a bounded cache, and a defined behavior when the service is unavailable.

A larger platform becomes relevant when approval workflows, percentage targeting, and long-lived audit history are requirements. LaunchDarkly, Unleash, and Flagsmith represent different product boundaries and deployment models; their existence does not make any one of them the right answer for this agent loop. Compare the capabilities that matter to the system: targeting semantics, SDK behavior during an outage, audit retention, export options, and whether decision metadata can join your trace data.

The deciding test is reproducibility. Given a run ID, tenant, flag snapshot version, and context, can another engineer reconstruct the branch? If not, the toggle is configuration without an audit trail.

What breaks first in production?

Stale snapshots are the common surprise. A worker can evaluate a valid rule that is no longer current. Return the snapshot version and expose refresh age as a metric. Then an operator can distinguish “the rule selected model tier B” from “this worker had not refreshed for 18 minutes.”

The second trap is cardinality. A metric label for every run ID will damage a metrics backend. Keep high-cardinality IDs in logs or spans; use metrics for bounded dimensions such as flag key, result, source, and outcome.

The third is partial failure. Decide whether an unavailable toggle service should use the last known snapshot, a declared default, or fail the agent request. The answer depends on risk. A review-required flag should fail closed if skipping review creates a policy problem. A cosmetic interface flag can usually use a safe default. Document this per flag, not in a single global rule.

Keep the failure policy beside the rule definition. Otherwise, the person changing a model-tier flag has to remember a separate operations document, and the fallback becomes tribal knowledge.

This design is intentionally limited. It does not provide percentage rollouts, a management UI, or a durable audit database. Those are separate requirements. The useful baseline is smaller: deterministic evaluation, bounded context, correlated telemetry, and a version that makes each cost-affecting branch explainable.

References

• OpenTelemetry, “Logs signal concepts”: https://opentelemetry.io/docs/concepts/signals/logs/
• OpenFeature specification: https://openfeature.dev/specification/
• LaunchDarkly documentation, feature flags overview: https://launchdarkly.com/docs/home/flags
• Unleash documentation, feature toggles: https://docs.getunleash.io/reference/feature-toggles
• Flagsmith documentation, feature flags: https://docs.flagsmith.com/