SigNoz 空白 AI 可觀測性面板：產品拆解

I’ve been building ReplayLens, a project around AI agent quality and regression reporting. That got me interested in an engineering question: how do we make complex technical systems easier to understand when something isn't working as expected?

While reading SigNozs public AI observability documentation I noticed a product-design question.

When a developer opens an observability dashboard and sees panels how quickly can they figure out why?

This is a documentation-based teardown. I have not reproduced a bug in a running SigNoz instance so I'm treating this as a hypothesis to investigate than a confirmed defect.

1. The details

SigNozs AI Observability features help developers inspect LLM and agent workloads, including token usage, cost, latency, errors time to first token and tool calls.

The documentation explains that these panels depend on telemetry arriving in the expected format.

For example:

gen_ai.request.model is used by LLM panels and the model filter.
gen_ai.usage.input_tokens and gen_ai.usage.output_tokens support token and cost panels.
gen_ai.tool.name is used by tool-call panels.
AI Observability requires traces; integrations that send metrics and logs do not appear in these views.

SigNoz also provides Attribute Mapping to normalize attributes emitted under names by instrumentation libraries.

Sources

These are engineering requirements. Standardized attributes make telemetry more consistent across frameworks and providers.

The question I want to explore is how directly the product helps a developer identify which requirement is missing.

2. The potential friction

Imagine instrumenting a LLM application and opening AI Observability to understand its token consumption.

The panels are empty.

Possible explanations include:

The application isn't sending traces.
Traces arrive,. The expected GenAI attributes aren't present.
The instrumentation uses attribute names.
Model pricing isn't configured, so cost calculations aren't available as expected.
A self-hosted installation is missing a required traces-pipeline processor.

Each situation calls for a fix.

The documentation describes the prerequisites and configuration options. My product hypothesis is that developers would benefit from seeing more of this information directly in the workflow.

An empty panel shouldn't force a developer to guess which layer is responsible.

I have not verified how every one of these cases currently appears in the product. That would require a test.

3. Three improvements I'd explore

A. A telemetry-readiness checklist

A small diagnostic panel could report whether the system has detected:

Incoming traces
LLM spans
The model attribute
Input and output token attributes
Model pricing configuration
Required processors for self-hosted setups

Each status could link to the appropriate setup instructions.

The goal would be to reduce the time between seeing a panel and identifying the next useful debugging step.

B. Informative empty states

Instead of a generic empty chart a panel could explain the requirement it needs.

For example:

No token usage data found

“We haven't found matching spans with the attributes this panel uses. Inspect a span. Review your attribute mapping.”

The exact message should depend on what the system has checked. It shouldn't tell someone an attribute is missing if that hasn't been verified.

C. A visible validation path

SigNozs Attribute Mapping documentation already describes testing mappings against a sample span and inspecting the resulting attributes.

I'd explore bringing that validation flow to the moment a developer encounters missing data:

Select a span.
Highlight expected attributes that're present or absent.
Preview the mapped result.
Explain which panels the resulting attributes support.
Send a span and verify the result.

The idea isn't to replace the existing configuration. It's to make the connection between configuration and outcome easier to understand.

4. How I'd validate the hypothesis

I'd test this against a running instance before filing an issue or calling it a bug.

I'd send a set of controlled examples:

A trace containing the expected GenAI attributes.
A trace missing token-usage attributes.
A trace using attribute names.
An application sending metrics and logs but no traces.
A self-hosted setup with the required processors configured correctly and incorrectly.

For each case I'd record the UI state, available diagnostics steps needed to resolve the problem and whether a developer could identify the cause without repeatedly consulting documentation.

That would establish whether this is an usability gap, an already well-handled case or an opportunity for further improvement.

5. What I like about the existing approach

I wouldn't change the underlying commitment to OpenTelemetry conventions.

Standard attribute names make it easier to compare telemetry across applications and attribute mapping addresses the reality that different instrumentation libraries can emit names.

The documentation also makes important prerequisites explicit.

I'd build on those strengths by making the diagnostic experience as clear as the setup guidance.

The takeaway

Developer tools need to do more than display the data when everything is configured correctly.

They should also help developers understand why the expected data isn't there.

For AI observability making the gap between instrumentation and useful dashboards easier to diagnose is a product question.

I'd be interested, in how the SigNoz team thinks about telemetry readiness and empty-state diagnostics.

References

This is a documentation-based product teardown. No live benchmark or confirmed bug reproduction is claimed.