[AI FinOps ①] 傳統 FinOps 難以應付 AI 成本

1. A Harness.io survey of approximately 700 global companies conducted in May and June found that 67% allocated at least $250,000 a month to AI, while nearly 20% had monthly AI budgets approaching $1 million. The bottom table shows survey responses identifying departments responsible for AI costs. (Source: Harness.io, 2026 State of AI in FinOps)

Artificial intelligence (AI) spending has become a growing concern for businesses as rapid adoption raises the risk of unexpected bills. This series examines AI FinOps strategies designed to manage AI costs in real time and reduce waste.

Generative AI costs are difficult to predict because companies must pay not only for model usage but also for inference, graphics processing units (GPUs), application programming interfaces (APIs) and AI agent execution. Traditional cloud cost management approaches alone cannot adequately control these expenses.

○ Cloud FinOps Struggles With AI’s Fundamental Characteristics

Without a systematic management framework, such variability frequently leads to unexpected budget overruns.

According to 2026 State of AI in FinOps, a survey of approximately 700 global enterprise IT leaders conducted by DevOps platform provider Harness.io, more than 67% of companies spend at least $250,000 a month on AI. Yet 73% of AI projects exceeded their budgets because of unpredictable AI workload charges.

Experts identify four major reasons traditional cloud FinOps falls short in AI environments: token-based pricing, multimodel environments, rapid model evolution, and accelerating code generation and agent workflows.

1. A Harness.io survey of approximately 700 global companies conducted in May and June found that 67% allocated at least $250,000 a month to AI, while nearly 20% had monthly AI budgets approaching $1 million. The bottom table shows survey responses identifying departments responsible for AI costs. (Source: Harness.io, 2026 State of AI in FinOps)

Unlike conventional cloud operations, where resource consumption tends to follow relatively predictable patterns, AI models operate probabilistically. Even identical prompts can generate responses that differ in content and length, resulting in different costs. This variability makes accurate cost forecasting difficult even when total usage is known.

Generative AI services also impose strict rate limits, such as tokens per minute (TPM) and requests per minute (RPM). Large capacity buffers needed to handle peak demand are particularly difficult to scale flexibly like conventional cloud resources because of hardware constraints. As a result, multistep AI agents must share limited capacity, while reasoning models may consume unpredictable numbers of tokens during each execution.

○ Variable AI Workloads and Complex Provider Pricing

Capacity allocation also differs according to performance requirements. Shared capacity at the department or team level provides pay-as-you-go flexibility but can lead to increased latency and capacity shortages during periods of heavy demand.

Provisioned capacity, by contrast, offers dedicated resources that ensure stable performance and low latency. However, providers differ significantly in throughput, contractual commitments and excess-usage policies. These include hard limits, which block further usage once a predefined budget or token or request threshold is reached, and spillover, or soft limits, which allow requests to continue while charging for excess consumption. Aligning these complex pricing arrangements is a major challenge.

Opaque token pricing further reinforces the need for AI FinOps.

Tokens, which determine AI usage charges, are less intuitive than conventional cloud billing metrics such as monthly CPU hours or storage capacity in gigabytes. Even identical text queries can produce substantially different token counts depending on the model and tokenizer. Other factors, including context length, region, quantization level and hosting arrangements, make accurate pricing even more difficult.

2. AI spending waste caused by a lack of cost visibility, according to Harness.io. One-third of approximately 700 respondents said identifying the cause of AI operational problems takes more than a week because of gaps in ownership and visibility, during which AI costs continue to accumulate. More than half of respondents, or 57%, said they encouraged “tokenmaxxing,” maximizing employees’ AI usage regardless of business outcomes or value. Harness.io estimated that 26% of monthly AI budgets are wasted as a result. (Source: Harness.io, 2026 State of AI in FinOps)

Tokens are sensitive to minor changes. Simply moving a comma in a prompt or switching model versions can significantly alter response length, model behavior and ultimately cost.

AI service providers sometimes release new model snapshots without advance notice, reflecting the rapid pace of AI development and deployment. This means cost management must also account for technical factors such as prompt engineering.

Anthropic, OpenAI, Google, SpaceXAI, Meta, Moonshot and ZAI are among the six or seven companies currently considered leading frontier model developers. Major upgrades to their frontier models typically occur about every six months, while intermediate releases, such as .5 versions, and minor upgrades arrive every one to three months. For example, GPT-6.1 was announced less than a month after GPT-6, while Claude Sonnet 5.5 followed version 5.0 after three months.

Even intermediate and minor upgrades can deliver substantial performance improvements, sometimes surpassing existing frontier models. This rapid technological progress can render state-of-the-art (SOTA) models outdated within months.

○ Token-Based Billing Adds Cost Volatility

AI FinOps must be updated frequently and in real time to keep pace with rapid changes in AI models.

Another challenge is that AI can consume thousands of expensive tokens even when a task fails. Repeated prompt debugging can also generate lengthy outputs with little value, accumulating additional costs. Unlike conventional cloud environments, enterprise AI operations therefore require strategies for detecting failures and controlling the resulting expenses.

The broader and more diverse range of service providers also complicates AI FinOps strategies. Generative AI providers include not only model developers but also cloud service providers.

3. Harness.io research shows that companies’ simultaneous use of multiple AI and cloud providers fragments billing systems and creates inefficiencies. AI also increases costs across infrastructure, software, including SaaS licenses such as copilots, and the broader operating environment, rather than model usage alone. (Source: Harness.io, 2026 State of AI in FinOps)

The same model may be available through its developer’s own hosted API as well as platforms such as Microsoft Azure, Amazon Web Services (AWS) and Google Cloud Platform (GCP). Pricing, regions, API endpoints and contractual terms can vary widely.

This creates a procurement environment more complicated than traditional cloud pricing. Open-weight models that companies can fine-tune themselves further expand the available options, increasing the complexity of cost management.

Maintaining AI service availability and implementing failover present additional challenges. Cloud services typically address outages through multiregion strategies, but an outage at a major AI provider can affect all models hosted on its platform simultaneously. Switching to an alternative service through model routing is more complicated because it may require prompt modifications and architectural changes, while performance and cost characteristics may also differ.

Similarities Between Cloud FinOps and AI FinOps

Traditional cloud FinOps and AI FinOps also share several fundamental characteristics.

Both emerged from the need to manage usage-based resources efficiently. Like cloud services, AI platforms commonly operate on pay-as-you-go pricing models, making cost forecasting, usage visibility, cost allocation and governance to prevent uncontrolled spending equally important. Just as abandoned cloud instances continue generating charges, unmanaged AI agents can incur unnecessary token costs. Both environments therefore require rigorous resource management.

Beyond usage-based pricing, the two fields also share similarities in dedicated capacity and commitment-based discounts. Like reserved instances and Savings Plans in cloud computing, AI providers offer lower unit prices in exchange for committed usage or provisioned capacity agreements to ensure performance. Companies need sophisticated forecasting and commitment management strategies to minimize waste and maximize discounts.

Model selection based on performance and cost efficiency is another shared principle. Just as selecting the appropriate cloud instance type or stock-keeping unit (SKU) is central to cloud FinOps, AI FinOps requires choosing models that match workload requirements. There is little reason to use an expensive frontier model for a task that a smaller model can handle, just as simple workloads do not require costly GPU instances. Both approaches require continuous right-sizing as requirements and large language models (LLMs) evolve, along with validation of lower-cost alternatives.

Both fields also require over-provisioning strategies to mitigate risks. Allocating spare resources helps address outages and performance issues. Just as cloud operators deploy standby instances across availability zones, AI operators need additional capacity to handle traffic spikes. The challenge is to balance service reliability against costs while maintaining resilience in multivendor environments.

Resource tracking through tagging is another common practice. Cloud resources are tagged to allocate costs and establish accountability, and similar tags can be applied to AI usage data, including API requests. This enables companies to track and allocate AI expenses by feature, product or team.

Automated usage quota controls, such as shutting down idle resources and setting API token limits, are also effective cost management strategies in both cloud and AI environments. Because of these similarities, organizations with mature cloud FinOps frameworks can apply comparable approaches to AI FinOps.