zlacker

[return to "LLM and Bug Finding: Insights from a $2M Winning Team in the White House's AIxCC"]
1. hqzhao+Mb[view] [source] 2024-08-16 21:23:19
>>garlic+(OP)
I'm part of the team, and we used LLM agents extensively for smart bug finding and patching. I'm happy to discuss some insights, and share all of the approaches after grand final :)
◧◩
2. simonw+7j[view] [source] 2024-08-16 22:31:38
>>hqzhao+Mb
What kind of LLM agents did you use?
◧◩◪
3. hqzhao+ak[view] [source] 2024-08-16 22:43:07
>>simonw+7j
Based on popular pre-trained models like GPT-4, Claude Sonnet, and Gemini 1.5, we've built several agents designed to mimic the behaviors and habits of the experts on our team.

Our idea is straightforward: after a decade of auditing code and writing exploits, we've accumulated a wealth of experience. So, why not teach these agents to replicate what we do during bug hunting and exploit writing? Of course, the LLMs themselves aren't sufficient on their own, so we've integrated various program analysis techniques to augment the models and help the agents understand more complex and esoteric code.

◧◩◪◨
4. simonw+dt[view] [source] 2024-08-17 00:56:11
>>hqzhao+ak
When you call these things “agents” what do you mean by that? Is this a system prompt combined with some defined tools, or is it a different definition?
[go to top]