zlacker

>>galnag+(OP)
I was quite stunned at the success of Moltbot/moltbook, but I think im starting to understand it better these days. Most of Moltbook's success rides on the "prepackaged" aspect of its agent. Its a jump in accessibility to general audiences which are paying alot more attention to the tech sector than in previous decades. Most of the people paying attention to this space dont have the technical capabilities that many engineers do, so a highly perscriptive "buy mac mini, copy a couple of lines to install" appeals greatly, especially as this will be the first "agent" many of them will have interacted with.

The landscape of security was bad long before the metaphorical "unwashed masses" got hold of it. Now its quite alarming as there are waves of non-technical users doing the bare minimum to try and keep up to date with the growing hype.

The security nightmare happening here might end up being more persistant then we realize.

>>Simian+0R
That's a bit of an understatement. Every single LLM is 100% vulnerable by design. There is no way to close the hole. Simple mitigations like "allow lists" can be trivially worked around, either by prompt injection, or by the AI just deciding to work around it itself (reward hacking). The only solution is to segregate the LLM from all external input, and prevent it from making outbound network calls. And though MCPs and jails are the beginning of a mitigation for it, it gets worse: the AI can write obfuscated backdoors and slip them into your vibe-coded apps, either as code, or instructions to be executed by LLM later.

It's a machine designed to fight all your attempts to make it secure.

>>0xbadc+eE1
ya... the number of ways to infiltrate a malicious prompt and exfil data is overwhelming almost unlimited. Any tool that can hit a arbitrary url or make a dns request is basic an exfil path.

I recently did a test of a system that was triggering off email and had access to write to google sheets. Easy exfil via `IMPORTDATA`, but there's probably hundreds of ways to do it.

>>cjonas+Sl2
Guys, I think we just rediscovered fascism and social engineering. Lets make the torment nexus on the internet!