“Bug-free, secure code is now attainable” is a nice idea. It just happens to be wrong.
That ambitious statement was one of more than 80 replies to Eric Raymond’s optimistic take on AI-assisted vulnerability discoveries. Because the corpus of human-written code is finite, Raymond argues, so are its defects. With “a million friendly robots” to help fix them, we’re well-positioned to out-AI the bad guys and secure our software. I’m all for that. The problem is treating this as a bug hunt we’re going to finish. Today’s tools are uncovering holes in yesterday’s code. Why assume tomorrow’s tools won’t do the same to software we consider secure today?
For those of us who have spent decades arguing for open source, this is an interesting development. Raymond popularized Linus’s law: “Given enough eyeballs, all bugs are shallow.” Making source available never guaranteed enough qualified eyeballs, and let’s be real, few people ever cared to look. AI is finally putting more of those eyeballs to work. They just happen to be robots.
The open source opportunity lies in what we choose to do next. I strongly believe we need open source today more than ever: A shared codebase gives successive generations of bug hunters somewhere to contribute fixes everyone can use. It’s also, as I recently argued in my All Things Open keynote, a way for us to collectively decide which software matters and why. When generating another codebase becomes easy, that choice matters more.
As I’ve written, open source code was never inherently more secure. Few people inspected it. The strength was that when someone found a problem, the code and the process were open enough to do something about it.
AI makes the discovery part more interesting. In November 2024, Google reported 26 vulnerabilities found using AI-generated fuzz targets in projects that had already undergone hundreds of thousands of hours of fuzzing. In March, Mozilla’s work with Anthropic uncovered 22 security vulnerabilities in Firefox, including 14 rated high severity. This is software that has received a lot of scrutiny, yet large language models were still able to scrutinize more deeply. That’s impressive. It’s also a strange foundation for declaring the end of software bugs. The lesson of these discoveries is that our confidence has run ahead of our ability to inspect the code. Better tools have exposed the gap, but they 100% have not established that no gap remains.
Raymond’s finite-code argument doesn’t get us around this. Even if we freeze a codebase and grant that its defects are finite, that doesn’t mean today’s tools can find them all, much less prove they have. Oh, and P.S.: We don’t freeze production software. Not really. We change code, upgrade dependencies, and connect systems their original developers never expected to meet. Our optimistic commenter on Raymond’s post acknowledged that bugs will remain in interactions between systems and their complex behaviors. That’s quite an exception. Massive, even. For an enterprise, those interactions are much of the software it actually depends on.
None of this makes the fixes worthless. Mozilla said all 22 security vulnerabilities were fixed by publication, helped by reports with small, reproducible test cases. We can become more secure without arriving at some permanent state of security. But someone still had to validate the findings and get repairs into software people could use. That’s where the open source argument gets stronger. Better tools will keep finding things we missed. A shared project lets us carry the fixes forward together, rather than each company paying to rediscover and repair the same problems in its own private version.
Readable software is just the start
There’s a related argument that AI-assisted reverse engineering makes source access irrelevant. That is, if a machine can reconstruct useful code from a binary, haven’t we effectively made everything open source? Well, no. Software running on someone else’s servers doesn’t necessarily give you a binary to inspect. And a plausible reconstruction isn’t automatically a faithful one. A September research preprint found that some LLM-decompiled functions passed the supplied tests while behaving differently on other legitimate inputs. Clean-looking code and the original program aren’t necessarily the same thing.
Even granting much better reverse engineering, a decompiler doesn’t hand you the project’s tests, design discussions, build environment, or history. It certainly doesn’t give you an open source license. Being able to read an implementation doesn’t automatically give you permission to modify and redistribute it.
Those rights matter because they let you do something with what you find. You can contribute a fix upstream or maintain a fork when necessary. A proprietary vendor can run excellent AI audits, too, and a common codebase isn’t unique to open source. The
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article: