zlacker

[return to "LLM and Bug Finding: Insights from a $2M Winning Team in the White House's AIxCC"]
1. hqzhao+Mb[view] [source] 2024-08-16 21:23:19
>>garlic+(OP)
I'm part of the team, and we used LLM agents extensively for smart bug finding and patching. I'm happy to discuss some insights, and share all of the approaches after grand final :)
◧◩
2. doctor+Uh[view] [source] 2024-08-16 22:20:02
>>hqzhao+Mb
Everyone thinks bug bounties should be higher. How high should they be? Who should pay for them?
◧◩◪
3. hqzhao+Bj[view] [source] 2024-08-16 22:35:54
>>doctor+Uh
It really depends on the target and the quality of the vulnerability. For example, low-quality software on GitHub might not warrant high bug bounties, and that's understandable. However, critical components like KVM, ESXi, WebKit, etc., need to be taken much more seriously.

For vendor-specific software, the responsibility to pay should fall on the vendor. When it comes to open-source software, a foundation funded by the vendors who rely on it for core productivity would be ideal.

For high-quality vulnerabilities, especially those that can demonstrate exploitability without any prerequisites (e.g., zero-click remote jailbreaks), the bounties should be on par with those offered at competitions like Pwn2Own. :)

◧◩◪◨
4. doctor+Eq[view] [source] 2024-08-17 00:10:13
>>hqzhao+Bj
It seems really hard for people to like, name some vulnerabilities, name some prices. I'm glad you are playing along. Which scenario makes more sense:

    The Punchline: Microsoft pays $10m for vulnerabilities like the kind used to exploit SolarWinds and the Azure token audience vulnerability.

    The Status Quo: Thousands of people pay CrowdStrike a total of billions of dollars, in exchange for urgent patching when vulnerabilities become known.
Okay, do you see what I am getting at? On the one hand, if you pay bug bounties, the bugs get fixed, and they sure seem expensive. But if you look into how much money is spent on valueless security theatre, it is a total drop in the bucket. But CrowdStrike hires security researchers!

So what should the prices really be? For which vulnerabilities? The SolarWinds issue is probably worth more than $10m, if people are willing to pay 100x more to CrowdStrike for nothing.

◧◩◪◨⬒
5. necove+JF[view] [source] 2024-08-17 06:00:18
>>doctor+Eq
It's not as simple: those billions of dollars are not just for this particular issue, or even just for security support.

It's also a difference between keeping a software engineer on staff and hiring a contractor as needed. One is cheaper for the company even if the hourly rate is higher.

The better question is how we can improve the overall security of the software we write, which this article is more focused on. But we understand that there will be bugs, and security bugs even, no matter how hard we try.

Even DJB (of qmail fame) and Knuth (of TeX and TAOCP fame) pay out bug bounties, and they heavily focus on software correctness over large feature sets.

[go to top]