跳到正文
DEV Community· Span Observer·· 2 小時前AI 評分52

Dynatrace 收購 Arize 對端到端 AI 可觀測性的影響,從原型到正式環境

End-to-end AI observability: what Dynatrace’s acquisition of Arize changes, from prototype to production

AI 導讀

Dynatrace 完成收購 Arize,雙方的目標是連接 AI 工程與平台營運,覆蓋 AI 生命週期。Arize AX 與 Dynatrace 平台仍可分開使用,Phoenix 維持開源,整合路線圖尚未公佈。文章提出以 OpenTelemetry 追蹤模型呼叫、評估輸出品質,並將 AI 訊號連結至基建及業務指標的實踐方法。

正文

AI observability means understanding not only whether an artificial intelligence application responds, but whether it responds correctly, at what cost and by which path. Dynatrace has completed its acquisition of Arize, the vendor behind the Arize AX platform and the open source Phoenix project: a good moment to set out what this topic really covers, from the first prototype through to large-scale operations.

Until now, two worlds have coexisted without really talking to each other. On one side, AI engineers trace their agents, run evaluations and compare experiments in tools designed for them. On the other, SRE and platform teams monitor services, infrastructure and user experience. In between, business teams see the effects, good or bad, without seeing the causes.

This article is the entry point to our series on AI observability. It describes the full lifecycle, proposes a concrete method to put it in place and points to more focused articles for each stage.

Why AI observability is more than application monitoring

Take a customer support assistant connected to a document base. One morning, it tells dozens of customers that the refund period is 30 days, when the policy changed to 14 days the week before. No HTTP errors, normal latency, perfect availability. From a classic monitoring point of view, everything is green. Yet, from the business’s point of view, the service is down.

That is the peculiarity of AI systems: they can fail silently, with a plausible and confident answer. An agent can also complete its task by taking a costly detour, calling the same tool five times or reading an outdated document. To diagnose these cases, you need signals that application telemetry alone does not provide: the agent’s trajectory, the retrieved context, the tools called, the tokens consumed and a measure of answer quality.

Conversely, AI telemetry alone is not enough either. A slow answer can come from a saturated vector database, a struggling GPU node or a network call to a model provider. Useful AI observability links both levels.

What Dynatrace’s acquisition of Arize changes

Arize targets AI engineers: tracing applications and agents, evaluations, datasets, experiments. Dynatrace targets the teams that run the digital environment as a whole. The stated aim is to connect these two disciplines to cover the entire AI lifecycle.

A few facts are worth remembering:

  • Arize AX and the Dynatrace platform remain available separately, and Phoenix remains an open source project.
  • Phoenix builds on OpenTelemetry and on OpenInference, the instrumentation conventions created by Arize for AI.
  • Dynatrace has said it intends to back Phoenix and contribute to the stewardship of OpenInference; for ongoing alignment with OpenTelemetry’s GenAI conventions, follow the official repositories of both projects.
  • The integration roadmap has not yet been published: Dynatrace says it will be detailed over the coming months.

For a team getting started, the practical consequence is simple: betting on open standards (OpenTelemetry, GenAI conventions) remains the least risky choice, whatever the final tool. We compare the open source options in our article on Langfuse and OpenLLMetry.

The four stages of AI observability, from prototype to production

Prototype: see what the agent does

From the first trials, trace every model call, every document retrieval and every tool called. These traces are used for debugging, but above all to build the first evaluation examples.

Pre-production: evaluate before you ship

A set of reference questions, evaluators (rules, comparison with an expected answer or LLM-as-a-judge) and a quality threshold built into the pipeline. A prompt or model change then becomes testable like any other code change.

Production: LLM traces, latency and costs

In operations, you follow the same traces, enriched with infrastructure context: latency per step, tokens per request, cost per conversation, provider errors. SREs add their service level objectives, as we explain in what AI workloads change for SREs.

Continuous improvement: detect drift

An AI system never stands still: a new model, a modified prompt, added documents, a replaced tool. Drift is detected by continuously evaluating a sample of real traffic. Every incident should produce a new example in the regression set.

In practice: building an AI observability loop

Step 1 — Map the use cases

List the applications and agents in production or in development, with for each one: the model used, the data sources, the authorised tools, the business process served and the team responsible. Without this inventory, you cannot know what to instrument first.

Step 2 — Instrument with OpenTelemetry

Add a span per model call with the attributes from the GenAI semantic conventions. The example below is deliberately simplified; client_llm stands for your own client.

from opentelemetry import trace

tracer = trace.get_tracer("assistant-support")

def repondre(question, documents):
    with tracer.start_as_current_span("chat assistant-support") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.request.model", MODELE)
        span.set_attribute("app.rag.nb_documents", len(documents))
        reponse = client_llm.generer(question, documents)
        span.set_attribute("gen_ai.usage.input_tokens", reponse.tokens_entree)
        span.set_attribute("gen_ai.usage.output_tokens", reponse.tokens_sortie)
        return reponse.texte

GenAI attribute names are still evolving: check which version of the conventions your libraries support. To follow a request from the interface down to the host, see also end-to-end OpenTelemetry with Dynatrace.

Step 3 — Define evaluators

Start small: three to five measurable criteria. For example, for the support assistant:

evaluateurs:
  - nom: reponse_fondee        # does the answer cite a retrieved document?
    type: llm_juge
  - nom: politique_a_jour      # is the cited document less than 30 days old?
    type: regle
  - nom: longueur_raisonnable
    type: regle
    max_tokens_sortie: 400
seuil_publication: 0.90

Step 4 — Link to SRE and business signals

Associate each AI trace with its service, its host or pod and the relevant business event (ticket resolved, case processed). This link is what lets you say whether a drop in quality comes from the model, the infrastructure or the data. Business-side measurement is covered in measuring AI ROI.

Step 5 — Feed the lessons back

Every poorly rated answer in production joins the regression set. Every incident leads to a new evaluator or test. The loop closes: production feeds development.

Best practices

  • Instrument from the prototype stage rather than on the eve of going live.
  • Track cost per successful task, not just total token cost.
  • Mask or exclude personal data from traced prompts and answers.
  • Continuously evaluate a sample of real traffic, not only before each release.
  • Version prompts, models and evaluation sets like code.
  • Keep a human in the loop for high-impact agent actions, a point covered in AI and day-to-day reliability.

Platform teams play a key role in making these practices the default, as shown in our overview of platform engineering in 2026.

Frequently asked questions

What is the difference between AI observability and model monitoring?

Model monitoring mainly watches metrics of the model itself. AI observability covers the whole chain: agent traces, context, tools, infrastructure, costs and business effects, with the ability to investigate a specific case.

Does Phoenix remain open source after the acquisition?

Yes. According to Dynatrace’s announcement, Phoenix remains available as open source and Arize AX remains a separate offering. Check the official repository for the project’s current status.

Should you wait for the Arize–Dynatrace integration before starting?

No. Instrumenting with OpenTelemetry and the GenAI conventions, building an evaluation set and linking traces to services are useful steps right now, whatever tool you choose.

Conclusion

AI observability is not one more dashboard: it is a loop that connects the building, the running and the business value of an AI system. The coming together of Dynatrace and Arize illustrates this convergence, but the method is available today: trace, evaluate, link, feed back. To dig deeper into the open foundations, see the Phoenix GitHub repository.

Further reading: Dynatrace completes acquisition of Arize to advance full-lifecycle AI observability (Dynatrace blog).

span.observer is an independent community blog, not affiliated with Dynatrace LLC or the projects mentioned. Trademarks belong to their respective owners. Tools change quickly: check commands and options in their official documentation.


Originally published on span.observer. French version: https://span.observer/fr/observabilite-ia-dynatrace-arize/.

來源:DEV Community · dev.to