AI 漏洞檢查令開源軟件更具價值

“Bug-free, secure code is now attainable” is a nice idea. It just happens to be wrong.

That ambitious statement was one of more than 80 replies to Eric Raymond’s optimistic take on AI-assisted vulnerability discoveries. Because the corpus of human-written code is finite, Raymond argues, so are its defects. With “a million friendly robots” to help fix them, we’re well-positioned to out-AI the bad guys and secure our software. I’m all for that. The problem is treating this as a bug hunt we’re going to finish. Today’s tools are uncovering holes in yesterday’s code. Why assume tomorrow’s tools won’t do the same to software we consider secure today?

For those of us who have spent decades arguing for open source, this is an interesting development. Raymond popularized Linus’s law: “Given enough eyeballs, all bugs are shallow.” Making source available never guaranteed enough qualified eyeballs, and let’s be real, few people ever cared to look. AI is finally putting more of those eyeballs to work. They just happen to be robots.

The open source opportunity lies in what we choose to do next. I strongly believe we need open source today more than ever: A shared codebase gives successive generations of bug hunters somewhere to contribute fixes everyone can use. It’s also, as I recently argued in my All Things Open keynote, a way for us to collectively decide which software matters and why. When generating another codebase becomes easy, that choice matters more.

The bug hunt doesn’t end

As I’ve written, open source code was never inherently more secure. Few people inspected it. The strength was that when someone found a problem, the code and the process were open enough to do something about it.

AI makes the discovery part more interesting. In November 2024, Google reported 26 vulnerabilities found using AI-generated fuzz targets in projects that had already undergone hundreds of thousands of hours of fuzzing. In March, Mozilla’s work with Anthropic uncovered 22 security vulnerabilities in Firefox, including 14 rated high severity. This is software that has received a lot of scrutiny, yet large language models were still able to scrutinize more deeply. That’s impressive. It’s also a strange foundation for declaring the end of software bugs. The lesson of these discoveries is that our confidence has run ahead of our ability to inspect the code. Better tools have exposed the gap, but they 100% have not established that no gap remains.

Raymond’s finite-code argument doesn’t get us around this. Even if we freeze a codebase and grant that its defects are finite, that doesn’t mean today’s tools can find them all, much less prove they have. Oh, and P.S.: We don’t freeze production software. Not really. We change code, upgrade dependencies, and connect systems their original developers never expected to meet. Our optimistic commenter on Raymond’s post acknowledged that bugs will remain in interactions between systems and their complex behaviors. That’s quite an exception. Massive, even. For an enterprise, those interactions are much of the software it actually depends on.

None of this makes the fixes worthless. Mozilla said all 22 security vulnerabilities were fixed by publication, helped by reports with small, reproducible test cases. We can become more secure without arriving at some permanent state of security. But someone still had to validate the findings and get repairs into software people could use. That’s where the open source argument gets stronger. Better tools will keep finding things we missed. A shared project lets us carry the fixes forward together, rather than each company paying to rediscover and repair the same problems in its own private version.

Readable software is just the start

There’s a related argument that AI-assisted reverse engineering makes source access irrelevant. That is, if a machine can reconstruct useful code from a binary, haven’t we effectively made everything open source? Well, no. Software running on someone else’s servers doesn’t necessarily give you a binary to inspect. And a plausible reconstruction isn’t automatically a faithful one. A September research preprint found that some LLM-decompiled functions passed the supplied tests while behaving differently on other legitimate inputs. Clean-looking code and the original program aren’t necessarily the same thing.

Even granting much better reverse engineering, a decompiler doesn’t hand you the project’s tests, design discussions, build environment, or history. It certainly doesn’t give you an open source license. Being able to read an implementation doesn’t automatically give you permission to modify and redistribute it.

Those rights matter because they let you do something with what you find. You can contribute a fix upstream or maintain a fork when necessary. A proprietary vendor can run excellent AI audits, too, and a common codebase isn’t unique to open source. The difference is who gets to participate in maintaining it and whether that work can continue independently of the original supplier.

So I wouldn’t write off source access. Better robot eyeballs make usable source more valuable. But the bigger opportunity has always been the project around it: people deciding what’s worth maintaining and sharing the work.

We still have to maintain software

PHP’s October 21 security release makes this painfully concrete. A developer identified signs of one underlying problem in 2024. Its potential exploitability was understood in principle, but there wasn’t a concrete attack path. Fixes involved difficult trade-offs in compatibility, performance, and third-party extensions. Limited expert capacity and a stalled consensus process kept the work from moving forward.

AI-assisted research subsequently demonstrated the problem’s full impact. The PHP Foundation and maintainers coordinated fixes across supported releases, and the Foundation acknowledged failures in resources and decision-making. More robot eyeballs helped make the danger concrete. They didn’t decide which compatibility costs to accept or organize the release. People did. Yes, people. We keep talking about automating people out of software development, while the work keeps demonstrating why we need people who understand the software and can make decisions about it.

We also need to stop indulging the silly rhetoric that AI obviates the need for shared code. “I can use Claude to build software perfectly tailored to my individual needs” sounds great until those needs include maintaining a database, operating system, or office suite for the next decade. Few enterprises want sole responsibility for all of that, however quickly an agent can produce the first version.

On the open source side, I’ve lobbied for bringing customer fixes into the main project for precisely this reason. A private patch solves today’s problem while leaving its owner responsible for carrying it through every future upgrade. An entirely private codebase makes that responsibility considerably larger. Cheap code generation doesn’t make ongoing maintenance free.

Consider an agent patching an authorization flaw in a library. In a shared project, that repair can become a regression test, a supported release, and an update for every downstream user. With a hundred private rewrites, a hundred teams have to work out whether they have the same problem and whether their fixes hold. AI may make that work cheaper. Choosing to repeat it a hundred times still seems like an odd definition of progress.

Shared code has costs, too. A widely deployed dependency can spread a flaw widely, and a fix in a repository protects nobody until users install it. That’s a reason to support maintainers and get serious about updates rather than assume public code takes care of itself.

Raymond is right that defenders need better tools. But the more capable those tools become, the stranger it seems to insist on replacing mature shared projects with a custom pile of software. If tomorrow’s robots will find bugs that today’s robots missed, I’d rather have a shared project ready to do something about them.