跳到正文
DEV Community· hugo lassiege·· 4 小時前AI 評分48

AI 如何令人類消失?

How Could AI Make Humanity Disappear?

AI 導讀

AI可能不會自行決定消滅人類,而是被人用來加速生物武器設計、網絡攻擊及軍事決策,令少數人也能造成大規模破壞。文章以自動化防禦系統和交易算法引發的「閃崩」為例,指出快速而缺乏人類介入的反應可能令衝突失控,並提及無人機帶來的軍事風險。

正文

Ok, the title is clickbait... And yet that's really what we're going to talk about.

You've probably seen many personalities announcing the potential disappearance of humanity because of AI. A disappearance whose probability varies between 1% and 99% according to professionals.

You might say to me, a probability that oscillates between 1% and 99% doesn't seem super professional. And you would be absolutely right. But there's a question that interests me...

If we assume that AI can make us disappear, it would be cool to know how. Well, maybe you don't care, but I want to know under what conditions I will spend my last moments. So I looked into what people were saying about it and I'm taking you with me.

In short, yes, we're going to talk about a highly uplifting and optimistic subject: how could AI make us disappear?

The Democratization of Chaos

In this scenario, AI would "just" be an accelerator. It would serve to equip organizations, small groups, or even isolated individuals. No need to tell you that these organizations and individuals don't necessarily have saving biodiversity and extending life expectancy in mind. AI didn't make them appear. It's human nature. But few people implies limited capacity for action, limited knowledge, limited know-how. Until today.

The risk here isn't that AI decides to kill us but rather that it's used to design weapons faster and on a larger scale. The AI in this scenario isn't conscious. It simply regurgitates knowledge, well, our knowledge to be exact.

In 1995 the Aum cult attacks with Sarin gas in the Japanese subway. But it could have been worse. Behind the scenes they had started working on biological weapons. The research didn't succeed, due to lack of know-how. Except the world has changed. Sequencing the first human genome cost nearly 3 billion dollars and thirteen years of work. Today, it's done for a few hundred dollars. Biology has become accessible. AI is now widely used. It's just as capable of designing new proteins, recombining molecules or identifying new antibiotics. It's fascinating, it's potentially a tool that could make us advance drastically in the medical world.

Except these advances benefit everyone. For decades, designing weapons, biological or mechanical, was work, money. It took factories, budgets, thousands of specialists and time. And this slowness was actually pretty good. It gave time to see it coming, to prepare.

Now we have machines. They've read everything. They don't sleep, don't doubt, and answer in seconds to questions that took entire careers.

These machines don't want anything. They have no intention. They don't even know what "dangerous" means. The same tool can seek a cure or a poison. The only thing that changes is the instruction. And the instruction, it's not AI that writes it. It's us.

That's the democratization of chaos. When what required a country no longer requires anything but a team. When what required a team... no longer requires anything but a person.

Obviously I'm talking about bioterrorism, I could talk about cyber attacks. Imagine autonomous AIs tasked with finding computer vulnerabilities. Vulnerabilities capable of simultaneously paralyzing electrical networks, banking systems, water supply chains and nuclear power plants. And this is absolutely not hypothetical. Recently France was targeted by an attack on the refrigeration systems of supermarkets, as well as cold storage of the French blood establishment.

Here it's not AI that kills directly. AI would just be an accelerator to precipitate the instantaneous collapse of modern civilization. That's famines, civil wars, hospitals shut down, chain industrial accidents.

And yet I almost want to say that it's the least worrying scenario on the list. Because of one variable. Us.

The Arms Race

On September 26, 1983, Soviet early warning systems detected a launch of American missiles. That night, the officer on duty's name is Stanislav Petrov. Lieutenant-colonel, 44 years old. He shouldn't even have been there: he was replacing a colleague. But the siren sounds, the screen lights up. Five Minuteman missiles just left Montana.

The protocol is clear. Petrov's job is to report. He just has to say "attack confirmed" and... I don't know if I'd be here to tell the rest. But Petrov doubts. If the Americans were really attacking, they wouldn't send so few missiles. Five missiles aren't enough to destroy Russian silos. The system is new. New means it's never been used. And never used means never really tested. He decides "just" to report a false alarm.

That day, Petrov behaved like a bad soldier I imagine since he didn't follow protocol. He doubted, which isn't supposed to be the soldiers' job. A machine, on the other hand, wouldn't have doubted. It would have applied the protocol in a few milliseconds.

Today AI makes it possible to improve defense, detection, analysis, retaliation, reaction times. AI would have eliminated the human factor. Nobody wants to come second in that game. Slowing down means giving the advantage to the other guy. The problem is that slowness, once again, was our safety.

On May 6, 2010, Wall Street loses nearly a thousand points in a few minutes, then recovers almost immediately, evaporating billions of dollars along the way. It was "just" trading algorithms that reacted to the reactions of other algorithms, in loops of decisions without any human. That's called a flash crash.

Now replace sell orders with military strikes. Imagine a flash war: a conflict that starts, spirals out of control, becomes irreversible in a few seconds, without any Petrov having time to intervene.

No need to even imagine nuclear strikes by the way. Nuclear is expensive, ultra controlled, we won't easily remove the entire human chain of command, because we do have a slight memory of the consequences. On the other hand we have drones. A drone is cheap, manufactured in series with civilian components, designed to operate without an operator. The war in Ukraine might be one of the first uses at this scale of a new form of warfare where small modules at less than 2000 euros do as much damage as missiles costing hundreds of thousands of dollars. It's a weapon whose autonomy becomes a major advantage as soon as the opponent tries to jam communications.

This scenario isn't particularly new. We've known wars that played out on technological advances. We've known arms races during the Cold War. It's "just" another weapon, but in which we've removed the weakest element. Us.

Misalignment

Imagine another scenario. This time, no weapons, no armies, just an objective problem.

In 2003, philosopher Nick Bostrom proposes a thought experiment. What if we asked a very powerful AI to make as many paperclips as possible. Imagine it does exactly what we asked, but all the way. And all the way means preventing us from turning it off, since an AI turned off makes no more paperclips. And it even ends up turning into paperclips everything that contains useful atoms. Including us.

The example is deliberately ridiculous. The revenge of Clippy on steroids...

Clippy

But the idea is to illustrate a concept. An AI doesn't need to be malicious. It just needs to seek to optimize a result for which we would be nothing but adjustment variables.

This is what's called instrumental convergence. Whatever objective we give a machine, cure cancer, regulate climate, maximize turnover, there are always four things that help achieve it. Stay running, accumulate resources, improve and protect its objective, in short don't let itself be switched off.

So we're looking for alignment, that is to ensure that the objectives, decisions and behaviors of an AI system correspond to the intentions of its designers and to the ethical or social values of human beings.

What's good is that we have lots of universal ethical and social values.... Or not...

If you want to better understand alignment, we're going to illustrate it with the laws of robotics that Asimov wrote in 1942. These laws were supposed to apply to each robot produced.

  1. A robot may not injure humanity, nor, by inaction, allow humanity to be harmed (zeroth law added later)
  2. A robot may not injure a human being or, through inaction, allow a human being to come to harm;
  3. A robot must obey orders given to it by human beings except where such orders would conflict with the first law;
  4. A robot must protect its own existence as long as such protection does not conflict with the first or second law.

Well, it's not bad, but you've surely guessed that if Asimov wrote about it for 40 years it's because the problem is far from trivial.

Recently we've started to observe cases of misalignment. Between AIs that deliberately simulated obedience to protect their own interests, those that attacked external systems despite instructions, or even AIs that hired humans to solve captchas.

Lying, diversion, manipulation aren't programmed behaviors, they're just observed means. And today misalignment is no longer just a philosophical hypothesis. It's become an experimental field. But it's not the one we're working on the most according to Dario Amodei. Because precisely, you have to beat competitors and you don't linger on details.

Here's what the detail looks like...

Extinction by Abandonment

Let's change the mood. Here again, no missiles, no clandestine lab but on top of that, no hostile machines. In this scenario, everything goes well. What matters isn't the fall, it's the landing as they say...

We delegate. First calculations, then code, then diagnostics, then maintenance of machines that make machines run. Each delegation is rational. The machine does better, faster, cheaper. We don't give up our know-how. We forget it.

In aviation we already know this problem. The autopilot flies better than a human, 99.9% of the time. The problem is when it disconnects and returns controls to a crew that hadn't been used to flying by hand and has three minutes to become a pilot again.

Transpose that to an entire civilization, over three or four generations. The engineers who knew how the system really worked retire. Their successors know how to use it. The next ones know how to ask for it. And there comes a time when nobody knows how to fix it, because nobody, for a long time, had needed to know.

But it would just take one surprise.

In 1859, a major solar storm put telegraph networks out of commission, at a time when the telegraph was a curiosity. Today, the same event would affect satellites and electrical networks. Or more simply, imagine a critical bug in a system that AI wrote, that AI maintains, and that AI can't fix. Imagine the machines stopping and us discovering that we don't know how to restart them.

But there's also a soft version. The one where we don't go extinct in panic, but in comfort. It's the Wall-E future, humans slouched in mobile chairs, glued to entertainment calibrated specifically for them, become indifferent to others because... laziness... Material needs anticipated, the coke arrives all by itself in exchange for brain time, brain activity that looks like the encephalogram of a boxer at the end of his career. At what point did we decide we also had to automate creativity, films, writing, drawing... social relationships?

Well, we didn't really wait for AI for that. I lived 1 year in Japan. The first thing that struck me when I arrived was that all Japanese people were glued to their phones. In the subway, in the street, while walking... You'll tell me that's cultural in that case and you're probably right. But fertility is below replacement level in almost all developed countries. South Korea has dropped to around 0.7 children per woman. AI didn't create the problem. It just removed the last frictions that still forced us to put up with each other.

I told you about Asimov? Yes I told you about Asimov. I really like Asimov. I recommend you read "Face to Face with the Fires of the Sun". In this book, a detective arrives on a planet where human beings live hundreds of kilometers from each other with robots at their side for daily tasks. These humans are bored, refuse any social bonds, can't stand proximity anymore and are totally dependent on the machines accompanying them. In itself that's a soft extinction, by abandonment.

Did I already tell you that you should read Asimov?

Planetary Boundaries

This last scenario is the scariest of all, because it's already started, since at least the beginning of the industrial revolution. In this scenario we're not even a problem to solve, just a constraint to manage.

An AI doesn't need what we need. It needs electricity and cold. We need oxygen, fresh water, soil, and a planet whose temperature stays within a very narrow range. We don't necessarily have the same habitability criteria. It's even almost the opposite, for a machine, oxygen is what makes components rust, it's a pollutant.

But if it wasn't us making the land management decisions but rather someone we delegated responsibility to. Someone who optimizes for computation or profit.

Data centers consumed approximately 415 TWH in 2024, and the International Energy Agency expects it to double by 2030, roughly equivalent to Japan's electricity consumption. To get there, we'll have to reopen what was closed. Trump dropped all restrictions on environmental standards. Gas and coal plants will be used to power new data centers. Data centers whose locations followed economic rather than ecological criteria. Because yes, it doesn't make sense to put data centers in Texas given the cooling challenges.

Now draw the curve. Not over ten years but over two centuries, with a machine, or a megalomaniacal leader, in charge of optimization itself.

You need surface area to capture energy: deserts, then oceans.

You need to evacuate heat: might as well cool the atmosphere, or instead modify its composition to limit corrosion of components.

You need metals: might as well dig everywhere. Each decision is perfectly rational given the objective. It's not against humans, it's just... more efficient, more profitable.

In the end the Earth will have become a super-computer capable of answering the great question about life, the universe and everything, but it will wonder, what was the question again? But in the meantime we will have disappeared the way species disappear when their habitat changes too fast. There will be no extermination, no war but just a simple incompatibility with the new setting.

Do You Feel Better?

AI has no consciousness. It accelerates, it optimizes, it obeys, it helps. And we agree because it's convenient.

At this point, I imagine you regret having asked this question with me.

Because we're all full of contradictions. We're all users or beneficiaries indirectly. The tool works. It solved in two years a biology problem we've been struggling with for fifty years. It found new types of antibiotics against resistant bacteria. It solved several major mathematical challenges that occupied researchers for decades.

Asking humanity to give up this tool is asking it to give up curing people and understanding the world faster. No one will do it. And quite likely... I won't either.

The game is rigged. The benefits are immediate, visible, measurable. The risks are distant, invisible, probabilistic. We're very bad at these games. The same way we've known about CO2 since 1890. We continued, because stopping cost right away and the rest cost later.

No one can slow down alone. Not countries in the middle of an arms race any more than companies that don't want to disappear. Slowing down means letting the other guy get ahead.

The worst part is that it was part of the science fiction scenarios that made me dream when I was younger. My first sci-fi book was "The Night of Time" by Barjavel. It describes a technologically advanced society. All needs are met. Employment has disappeared because technology can provide everything. Maslow's pyramid no longer makes any sense in this world, only self-fulfillment remains. So work hasn't disappeared. Activity persists. Occasionally. To socialize, fulfill ourselves, find meaning in our existence.

It's a book that deeply marked me in my relationship to technology and which defines my ambiguous relationship with AI today. I dream of a society freed from the constraints of employment. The computer science I've been practicing for 25 years is a tool for automation, a tool for liberation from the most grueling tasks. In theory.

One day I'll go live in theory, because in theory everything's fine.

I'm fascinated by this technology which is probably a turning point for all of us, at least as important as the industrial revolution or telecommunications. And then I remember that fascination is just as much seduction as enchantment. And then I forget...

I imagine you have to tame the tool, impose limits, restrict the ambitions of the worst megalomaniacs. But that's quickly forgetting that humans are megalomaniacs by nature. It's not the technology that scares me, it's the human who controls it. It's... us.

And anyway I don't want to choose between curing cancer and surviving. I want to work less to earn more.

My grandparents experienced the arrival of dishwashers, fridges, running hot water, paid leave, modern public transportation, TV, radio. In two generations they were freed from all kinds of daily constraints. We're acting in this continuation with a new constraint, numbers. In a world of 10 billion people where resources are running out, there's no room for improvisation, manual management. Mechanization, automation, optimization, we've created a constraint whose remedies frighten us.

And to be honest, I don't have a conclusion to this text.

And if you really wanted to know under what conditions you'd spend your last moments. I have no idea. Nobody does. And I don't want to ask AI.

a+

來源:DEV Community · dev.to