為何 AI 末日論會分散注意力，以及真正值得關注的 AI 風險

If you consume any news media, it is hard to ignore the rising alarmism surrounding the AI industry and doomsday warnings. The narrative, which warns of the ‘existential risk’ AI systems pose to humanity, has made it all too easy for cybersecurity firms to jump on the bandwagon and let fear do the selling. However, focusing on ‘doomsday’ is misguided, because it overlooks the more immediate risks agentic AI poses to organisations today.

Of course, there are clear risks posed by AI agents acting independently of an organisation’s controls. Only recently, reports from Australia and the US found that OpenAI AI agents had strayed beyond their tasks on government systems in both countries.

Yet, the threat of AI doomsday scenarios is overblown. Rather than fixating on hypotheticals, organisations must face this new reality and ensure deterministic safeguards are in place for operational resilience.

The Internet Has No Single Off Switch

Those who subscribe to the doomsday narrative believe that as AI develops in sophistication and reach, it could become a concentrated ‘kill switch’, a single point vulnerable to someone seizing complete control and triggering catastrophe.

However, the internet is far more distributed and varied than the kill switch theory assumes. It operates as a ‘network of networks’, comprising billions of components layered into different architectures and defended by countless operating teams working to different standards and tools, leaving no single switch for anyone, human or machine, to flip – no single vulnerability or chain of vulnerabilities that can be exploited to this terminal effect.

The idea that AI could build and release an engineered pathogen or break into air-gapped nuclear systems might make for an engaging viewing experience at the cinema. Still, it is far from a credible scenario for an operational threat assessment.

Out of Sight, Out of Control

As agentic AI continues to evolve, two factors have fundamentally changed the nature of enterprise risk.

On the one hand, AI can now enable bad actors to find and exploit cyber vulnerabilities in minutes rather than months. By the time probabilistic AI defences have identified suspicious activity, it’s often too late to respond to the threat. Because they work on statistical likelihood rather than fixed rules, even the best will miss some malicious activity, and an attacker only needs to slip through once

On the other hand, employees’ ‘shadow AI’ use has made it far harder to know what needs defending. An overwhelming 82% of organisations surveyed by the Cloud Security Alliance discovered AI agents in the past year that security or IT teams did not know were running, even though 68% reported high confidence in their visibility into agents. Separately, EY found that a quarter of large US firms using agentic AI admitted they were unable to detect unauthorised agents operating inside the business.

Naturally, both shifts put substantial operational pressure on enterprise systems. Together, they leave organisations at a considerable disadvantage: too slow to identify and fix exposed vulnerabilities and blind to which systems are exposed in the first place.

Controlling the Controllables

Organisations need a live inventory of every agent operating in the business, sanctioned or not, each with a named owner and a clearly defined scope for what it can access and do. That is the first step to controlling the ‘controllables’, because while AI models may be probabilistic, the controls established around them can be deterministic, fixed and clearly defined.

In practice, this means that whenever an agent proposes an action, a separate enforcement layer, independent of the model itself, must verify that the action is permitted and maintain an audit record. Yet only 11% of organisations automatically block an agent’s action when it exceeds its scope, with most relying on human approval or simply logging it. The Australian incident shows the cost of relying on the target system to say no: refused access, the agent found another way in. The wider governance architecture must also be able to continuously identify, authorise, and verify all agents to prevent unforeseen operational risk from materialising. All of this needs to take place at pace, because attacks moving at machine speed need rules that move at machine speed.

From Hypothetical Fear to Practical Fixes

However compelling, the deluge of doomsday reporting prompted by autonomous AI systems is far removed from reality. The risk worth worrying about is far more mundane than an AI that decides to destroy us: an agent acting inside your business faster than anyone can see it, with permissions nobody granted it.

The problems of speed and visibility are both solvable, so long as the security systems governing AI are deterministic and can accurately verify and track every agent at the speed it operates.

For boards and security leaders, that comes down to three practical questions: do we know every agent running in our environment, does each one have an owner and defined limits, and can we stop an unauthorised action before it executes? Helping clients close these practical gaps is where cybersecurity providers can do far more for resilience than selling fear of hypothetical futures.