智能體數據存取：如何按來源選擇複製或聯邦查詢

Your agent pilot will not wait for the warehouse program to finish. If the agent can reach only a subset of your systems, choose that subset per source on purpose. Whatever happened to land in the lake last quarter is not a plan. Jitender Aswani puts the scale of the problem in plain terms: a traditional enterprise runs anywhere from 52 to 200 data sources, and the count keeps climbing with every SaaS tool and cloud service.

Centralization was a reasonable bet when data grew slower than your ability to move it. That balance flipped. Moving everything does not end the work; it changes it into governance load and pipeline maintenance while schemas keep shifting. Data federation is the other path: leave data where it lives and put a query layer on top that can reach across sources on demand. Federation addresses reach. Authority, identity and action policy on those sources are still separate work.

Agents change the math again. They run in reason-and-act loops, firing queries fast, including unbounded ones, when an answer is missing. Partial access does not make them pause; it makes them query harder. Aswani described Starburst’s internal FinOps agent, which could query AWS spend data but not the Azure and GCS data. Asked for spend by cloud provider, it kept querying instead of admitting the gap. That pattern is why I would decide copy vs federate per source before you wire tools.

When should a source stay query-in-place?

Federate when copying would buy you stale data, broken pipelines, or a moving target you do not own.

Jitender Aswani argues federation is the model that scales as source count and entropy grow, while centralization keeps chasing the next connector.

So, up until a point, centralization works fine. But as the data volume continues to explode, the ETL tech debt starts to really accumulate.

Jitender Aswani, SVP of Engineering and Security at Starburst

His assessment of full centralization is blunt: governance problems and continuous ETL challenges compound once everything has been moved.

When is a central copy still worth it?

Copy into a warehouse, lake, or curated mart when you control the pipeline, you need repeated heavy scans on a stable shape, and the agent workload is predictable.

Data sovereignty covers more than keeping data in one country. Teams also want to choose where a workload runs, move it, and pull it back on demand. That pushes architectures that can shift across hybrid and multi-cloud setups instead of locking one provider forever. If sovereignty rules say a copy must live in region A, treat that as a hard row in your table before anyone writes the agent prompt.

What breaks if you only fix storage, not meaning?

Reach without context produces confident wrong answers. A knowledge graph captures entities and relationships. It does not say whether “customer” means account or opportunity, whether fiscal year starts in February or July, or how “cohort” differs by team. A context graph layers those business rules and synonyms on top so agents interpret data the way the business does.

You can federate storage and still fail if context stays in Confluence, Google Docs, and people’s heads. Plan where context lives with the same discipline as where bytes live: federate scattered rules when they change often and sit with domain owners; centralize when one team maintains a canonical metric definition agents must share.

How do you wire agents without a token bonfire?

Declare coverage in the tool contract. List which sources each agent can query, what metrics definitions apply, and what to do when data is missing (stop and report, do not loop).

Sketch: source registry the orchestrator checks before calling SQL or MCP tools.

# Sketch only, not production code
from dataclasses import dataclass
from enum import Enum

class AccessMode(Enum):
    FEDERATED = "query_in_place"
    REPLICA = "central_copy"

@dataclass
class SourcePolicy:
    name: str
    mode: AccessMode
    reachable: bool
    freshness: str  # "live" | "batch_lag_hours"

REGISTRY = [
    SourcePolicy("aws_billing", AccessMode.FEDERATED, True, "live"),
    SourcePolicy("gcs_billing", AccessMode.FEDERATED, False, "live"),
]

def agent_preflight(question: str) -> str | None:
    missing = [s.name for s in REGISTRY if not s.reachable]
    if missing and "by cloud" in question.lower():
        return f"Cannot answer: no access to {', '.join(missing)}"
    return None

When Starburst shipped an MCP server, query volume spiked and customers scaled compute behind it. Agents do not budget queries the way a human analyst might.

And sometimes I've seen that like after 30 minutes, they're still doing it. And I know you are wasting my tokens. Just stop. I sent you on a wild goose chase and you did go on that wild goose chase.

Jitender Aswani, SVP of Engineering and Security at Starburst

I would expect preflight checks, explicit partial-access messages, and guardrails on retry loops to cost less than letting the model “work harder” with half the data.

For more on how enterprises are threading federation, context, and agent traffic together, see the Chain of Thought conversations on enterprise AI.

Checklist

• Inventory sources (expect dozens, not five) and mark each as federate or replicate using freshness, sovereignty, change rate, and who owns the system.
• Publish reachable sources and metric definitions to the agent layer; never assume it will infer missing clouds or schemas.
• Add a context plan (catalog plus business rules) beside your storage plan so “churn” means one thing in the answer.
• Load-test agent query paths before broad rollout; reason-act loops multiply traffic fast.
• Treat federation as reach. Identity and action controls stay a separate design pass.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.