Kitaru 的 Agent Harness：ZenML 如何運用 MLOps 基礎元件構建持久、可重播的智能體系統

Most agent frameworks treat execution as ephemeral scripts. You run a loop, it calls tools, maybe it logs to a file, then it exits. ZenML's new Kitaru project takes a different approach: wrap agents in the same durable infrastructure you'd use for batch ML pipelines. That means versioned artifacts, replayable execution graphs, and observability hooks borrowed from MLOps tooling.

Hamza Tahir, ZenML's founder, discussed the project on the Practical AI podcast. The core idea is simple: if you can version a trained model and replay its training run, you should be able to version an agent's state and replay its decision path. The gap between agent demos and production systems is mostly plumbing, and MLOps already solved half of it.

What an Agent Harness Actually Does

A harness is not a framework. It's a runtime wrapper that sits between your agent code and the execution environment. Think of it as a sidecar that captures state transitions, tool invocations, and decision points without forcing you to rewrite your agent logic.

Core responsibilities:

State capture: Snapshots agent memory, tool outputs, and intermediate reasoning steps
Replay semantics: Reconstructs execution from a checkpoint, replaying tool calls or skipping them with cached results
Observability hooks: Emits traces, logs, and metrics in a format your existing monitoring stack understands
Artifact versioning: Treats prompts, tool schemas, and model weights as immutable artifacts with lineage

The harness doesn't dictate your agent architecture. You can run ReAct loops, plan-and-execute patterns, or custom state machines. The harness just ensures you can inspect, debug, and reproduce what happened.

MLOps Primitives That Transfer to Agents

ZenML built its reputation on pipeline orchestration for batch ML. Kitaru ports three primitives to agent execution:

1. Pipeline Abstraction

In MLOps, a pipeline is a directed acyclic graph of steps. Each step consumes artifacts, runs code, and produces new artifacts. Agents break this model because they loop and branch based on runtime conditions.

Kitaru's solution: treat each agent iteration as a pipeline step. The loop becomes a meta-pipeline that spawns child pipelines. You get the same artifact tracking and caching semantics, but the graph can grow dynamically.

2. Artifact Stores

Every tool call result, every LLM response, every state transition gets written to an artifact store. This is not just logging. Artifacts are immutable, versioned, and queryable. You can diff two agent runs to see where their decision paths diverged.

3. Caching and Replay

If you replay an agent run and the inputs haven't changed, Kitaru can skip expensive tool calls and reuse cached results. This is critical for debugging. You don't want to re-run a $50 API call just to test a downstream logic branch.

MLOps Primitive	Batch ML Semantics	Agent Semantics	Kitaru Adaptation
Pipeline	Static DAG	Dynamic loop with branches	Meta-pipeline spawns child steps per iteration
Artifact	Model weights, datasets	Tool outputs, LLM responses, state snapshots	Immutable, versioned, diffable
Caching	Skip training step if inputs match	Skip tool call if context matches	Content-addressed cache with replay flag
Observability	Metrics, logs, lineage	Traces, decision logs, tool call spans	Unified trace format for existing APM tools

Architecture: How Kitaru Wraps Agent Execution

Kitaru doesn't replace your agent framework. It wraps it. You keep using LangChain, AutoGPT, or custom code. The harness intercepts execution at three points:

1. Initialization

The harness loads the agent's configuration (model, tools, prompts) from a versioned artifact store. It sets up observability hooks and initializes the replay cache.

2. Execution Loop

Each iteration of the agent loop becomes a pipeline step. The harness captures:

Input state (memory, context, previous tool results)
LLM call (prompt, response, token count, latency)
Tool invocations (parameters, outputs, errors)
Output state (updated memory, next action)

3. Termination

The harness writes a final artifact containing the full execution trace, all intermediate states, and a replay manifest. You can load this artifact later to resume or debug.

from kitaru import AgentHarness
from your_agent import MyReActAgent

# Wrap your agent in the harness
harness = AgentHarness(
    agent=MyReActAgent(),
    artifact_store="s3://my-bucket/agents",
    enable_replay=True,
    cache_tool_calls=True
)

# Run the agent (harness captures everything)
result = harness.run(
    task="Find the cheapest flight to Berlin",
    max_iterations=10
)

# Later: replay from a checkpoint
harness.replay(
    run_id="abc123",
    from_step=5,
    override_tool_results={"search_flights": cached_results}
)

Replay Semantics: What Gets Snapshotted

Replay is not just re-running the same code. It's reconstructing the exact decision path the agent took, then optionally diverging at a specific step.

What Kitaru snapshots:

Prompts and responses: Full text, not just embeddings
Tool call parameters: Exact arguments passed to each tool
Tool outputs: Raw results, not just success/fail flags
Agent memory: Vector store state, conversation history, working memory
Model weights: If you're fine-tuning, the exact checkpoint used
Environment state: API versions, tool schemas, external service responses

Replay modes:

Full replay: Re-execute everything, useful for validating determinism
Cached replay: Skip tool calls, use stored results, fast debugging
Divergent replay: Start from step N, override one input, see what changes

Observability: Traces, Not Just Logs

Kitaru emits OpenTelemetry traces. Each agent iteration is a span. Tool calls are child spans. LLM requests are nested spans with token counts and latency.

This plugs into your existing observability stack. If you're running Jaeger, Honeycomb, or Datadog, you get agent traces alongside your API traces. You can correlate an agent's decision with the downstream services it called.

Key metrics Kitaru tracks:

Iterations per task (how many loops before termination)
Tool call latency (which tools are slow)
LLM token usage (cost per task)
Cache hit rate (how often replay skips work)
Error rates (which tools fail most often)

Failure Modes and Mitigation

Agent harnesses introduce new failure surfaces:

Failure Mode	Symptom	Mitigation
Artifact store latency	Slow agent startup, high write latency	Use local cache with async upload
Replay divergence	Replayed run produces different results	Snapshot external API responses, not just tool outputs
State explosion	Artifact store fills with GB of traces	Prune old runs, compress snapshots, sample traces
Cache invalidation	Stale tool results break replay	Content-address cache keys, include tool version in hash

The biggest risk is assuming replay will always be deterministic. If your agent calls a live API that returns different results, replay will diverge. Kitaru can snapshot API responses, but you need to decide whether that's acceptable for your use case.

When to Use an Agent Harness

Good fit:

You're running agents in production and need to debug failures
You want to version agent behavior like you version models
You need to replay agent runs for compliance or auditing
You're building agent fleets and need centralized observability

Bad fit:

You're prototyping and don't care about durability yet
Your agents are stateless request-response systems (just use API logs)
You can't afford the storage overhead of full execution traces
Your agents run in environments without persistent storage

Technical Verdict

Use Kitaru if you replay agent runs at least twice per week for debugging, your artifact storage budget is under $500 per month, or you need compliance audit trails showing exact decision paths. It's a strong fit when agents run 10+ iterations per task and you're already using ZenML or similar MLOps tooling. The harness overhead (5-15% latency, 2-10 GB storage per 100 runs) pays off when you can skip re-running expensive tool calls during debugging.

Avoid it if your agents are stateless request-response systems where standard API logging suffices. Skip the harness if you run agents in ephemeral containers without persistent storage, or if you're still in prototype phase with fewer than three production agents. The replay infrastructure adds complexity that only matters when you're debugging non-deterministic failures or need to reconstruct past decisions for auditing.

The decision hinges on replay frequency and storage cost. If you're not looking at old agent runs to understand failures, you don't need the harness. If you are, and you can afford 100-500 MB per agent run, Kitaru gives you the same debugging experience you'd expect from batch ML pipelines.