For vendor-specific software, the responsibility to pay should fall on the vendor. When it comes to open-source software, a foundation funded by the vendors who rely on it for core productivity would be ideal.
For high-quality vulnerabilities, especially those that can demonstrate exploitability without any prerequisites (e.g., zero-click remote jailbreaks), the bounties should be on par with those offered at competitions like Pwn2Own. :)
Have you tested your CRS on weekend CTFs? I’m curious how well it’d be able to perform compared to other teams
Our idea is straightforward: after a decade of auditing code and writing exploits, we've accumulated a wealth of experience. So, why not teach these agents to replicate what we do during bug hunting and exploit writing? Of course, the LLMs themselves aren't sufficient on their own, so we've integrated various program analysis techniques to augment the models and help the agents understand more complex and esoteric code.
We haven't tested it yet. Regarding CTFs, I have some experience. I'm a member of the Tea Deliverers CTF team, and I participated in the DARPA CGC CTF back in 2016 with team b1o0p.
There are a few issues that make it challenging to directly apply our AIxCC approaches to CTF challenges:
1. *Format Compatibility:* This year’s DEFCON CTF finals didn’t follow a uniform format. The challenges were complex and involved formats like a Lua VM running on a custom Verilog simulator. Our system, however, is designed for source code repositories like Git repos.
2. *Binary vs. Source Code:* CTFs are heavily binary-oriented, whereas AIxCC is focused on source code. In CTFs, reverse engineering binaries is often required, but our system isn’t equipped to handle that yet. We are, however, interested in supporting binary analysis in the future!
The Punchline: Microsoft pays $10m for vulnerabilities like the kind used to exploit SolarWinds and the Azure token audience vulnerability.
The Status Quo: Thousands of people pay CrowdStrike a total of billions of dollars, in exchange for urgent patching when vulnerabilities become known.
So what should the prices really be? For which vulnerabilities? The SolarWinds issue is probably worth more than $10m, if people are willing to pay 100x more to CrowdStrike for nothing.
It's also a difference between keeping a software engineer on staff and hiring a contractor as needed. One is cheaper for the company even if the hourly rate is higher.
The better question is how we can improve the overall security of the software we write, which this article is more focused on. But we understand that there will be bugs, and security bugs even, no matter how hard we try.
Even DJB (of qmail fame) and Knuth (of TeX and TAOCP fame) pay out bug bounties, and they heavily focus on software correctness over large feature sets.