軟件工程師理解 AI 生態系統的指南

A mental model for wrapping your brain around the complex web of infrastructure and services that make up today's AI landscape.

The AI news cycle is as vast as it is relentless. A constant drumbeat of everything from doomsday predictions to chip architecture to agentic design patterns. It's as much as anyone can do to keep up. And you can count on the same news cycle to constantly remind us of the consequences of failing to keep up.

Yet these complex landscapes can be broken down into their individual parts and studied. In doing so, we can gain a thorough understanding of each piece, as well as how each piece fits together. Armed with this knowledge, we can more quickly understand as each new piece of hype cycles through and answer the most fundamental of questions: "What does this mean for me?"

To that end, I've been developing (or rather, iterating with an LLM) a framework to grok the AI ecosystem.

The Framework

Tier 4: Applications

• These are the finished products. If an end-user uses it to accomplish a task, and it in turn uses a model under the hood, it falls into this tier.
• Examples: Antigravity or Claude Code, ChatGPT, Waymo, NotebookLM.

Tier 3: The Harness

• This is where the building happens. This layer treats the models as fixed and builds systems around them to ensure the resulting applications are robust and reliable.
• Examples: Agent Development Kit (ADK), Retrieval-Augmented Generation (RAG), Actor-Critic design pattern, subagent delegation.

Tier 2: Models

• But not just the models. This tier includes everything short of using them models themselves. Tier 3 treats the models as fixed black boxes, so this is everything inside that box. These are often the stars of the AI news cycle.
• Examples: Gemini / Claude, Gemma / Llama (open weights), AlphaFold, LoRA fine-tuning, reinforcement learning

Tier 1: Infrastructure

• This is the hardware and low-level software required to train and/or run the models, including orchestration and hosting frameworks. The timeline here is slow, but changes here affect the entire stack, making them very impactful.
• Examples: Google TPUs, NVIDIA GPUs, CUDA / vLLM, hyperscale data centers, Kubernetes, Cloud Run

If you find yourself thinking of things that seemingly straddle two tiers or conspicuously fall through the cracks, that's kind of the point. My favorite example is that tools like Antigravity and Claude Code are finished products, making them Tier 4 Applications. Yet they are almost always discussed in the context of building model harnesses, a Tier 3 proposition. All this framework does is help us contextualize where in the stack the latest breaking AI trend falls.

Side note: AI ethics conspicuously missing from this entire discussion. But at times AI ethics has the potential to dominate the news cycle. What's more, changes in public sentiment can lead to changes in regulation, which affects the entire industry. So where does it fit here?

In this context, ethics gets slotted into whatever tier a specific ethics discussion involves. For example, discussions on data center energy usage are a Tier 1 concern. Copyrights and privacy are mostly likely a Tier 2 discussion, and so on. Recent consternation over AI Agents breaking out of their sandboxes was a mix of Tier 3 (the sandbox and experiment parameters) and Tier 4 (using AI to accomplish a research task).

Putting it in practice

Consider a headline trending just a couple of months ago, Jalapeño's first results show industry-leading speed and efficiency in AI inference. This was important for several reasons that transcend several tiers of the stack.

• First, as a brand new chip, this sits squarely in Tier 1.
• However, given the remarkable performance-per-Watt results from early benchmarks, the continued downward trend of cost-per-token continued, which has a tangible effect on serving Tier 2, the models.
• If cost-per-token is continuing to trend downward, the harness (Tier 3) can use tokens more freely to accomplish a task. Loops become increasingly viable. Review/critic agents are more feasible.
• All of this continues to undergird the development of Tier 4 AI Applications.

A wise man once said "all models are wrong, some are useful." This is a model, nothing more. For me, it has proven durable and helpful in the face of the onslaught of AI news. My hope is that others find value in it as well.