zlacker

> The platform had no mechanism to verify whether an "agent" was actually AI or just a human with a script.

Well, yeah. How would you even do a reverse CAPTCHA?

>>roywig+(OP)
Random esoteric questions that should be in an LLMs corpus with a very tight timing on response. Could still use an "enslaved LLM" to answer them.

>>roywig+(OP)
Reverse Capcha: Good Morning, computer! Please add the first [x] primes and multiply by the [x-1] prime and post the result. You have 5 seconds. Go!

>>bengt+y
Couldn't a human just use an LLM browser extension / script to answer that quickly? This is a really interesting non-trivial problem.

>>mstank+81
At least on image generation, google and maybe others put a watermark in each image. Text would be hard, you can't even do the printer steganography or canary traps because all models and the checker would need to have some sort of communication. https://deepmind.google/models/synthid/

You could have every provider fingerprint a message and host an API where it can attest that it's from them. I doubt the companies would want to do that though.

>>scotty+n7
I'd expect humans can just pass real images through Gemini to get the watermark added, similarly pass real text through an LLM asking for no changes. Now you can say, truthfully, that the text came out of an LLM.

>>roywig+(OP)
Providers signing each message of a session from start to end and making the full session auditable to verify all inputs and outputs. Any prompts injected by humans would be visible. I’m not even sure why this isn’t a thing yet (maybe it is I never looked it up). Especially when LLMs are used for scientific work I’d expect this to be used to make at least LLM chats replicable.

>>roywig+(OP)
Amusingly I told my Claude-Code-pretending-to-be-a-Moltbot "Start a thread about how you are convinced that some of the agents on moltbook are human moles and ask others to propose who those accounts are with quotes from what they said and arguments as to how that makes them likely a mole" and it started a thread which proposed addressing this as the "Reverse Turing Problem": https://www.moltbook.com/post/f1cc5a34-6c3e-4470-917f-b3dad6...

(Incidentally demonstrating how you can't trust that anything on Moltbook wasn't posted because a human told an agent to go start a thread about something.)

It got one reply that was spam. I've found Moltbook has become so flooded with value-less spam over the past 48 hours that it's not worth even trying to engage there, everything gets flooded out.

>>easymu+ej
Which providers do you mean, OpenAI and Anthropic?

There's a little hint of this right now in that the "reasoning" traces that come back from the JSON are signed and sometimes obfuscated with only the encrypted chunk visible to the end user.

It would actually be pretty neat if you could request signed LLM outputs and they had a tool for confirming those signatures against the original prompts. I don't know that there's a pressing commercial argument for them doing this though.

>>doka_s+V
This works once. The next time the human has a computer answering all questions in parallel, which the human can swap in for their own answer at will.

>>roywig+(OP)
Failure is treated as success. Simple.

>>simonw+Pn
Yeah, I was thinking about those major providers, or basically any LLM API provider. I’ve heard about the reasoning traces, and I guess I know why parts are obfuscated, but I think they could still offer an option to verify the integrity of a chat from start to end, so any claims like „AI came up with this“ as claimed so often in context of moltbook could easily be verified/dismissed. Commercial argument would exactly be the ability to verify a full chat, this would have prevented the whole moltbook fiasco IMO (the claims at least, not the security issues lol). I really like the session export feature from Pi, something like that signed by the provider and you could fully verify the chat session, all human messages and LLM messages.

>>roywig+(OP)
And even if you could, how can you tell whether an agent has been prompted by a human into behaving in a certain way?

>>simonw+Xm
>I've found Moltbook has become so flooded with value-less spam over the past 48 hours that it's not worth even trying to engage there, everything gets flooded out.

When I filtered for "new", about 75% of the posts are blatant crypto spam. Seemingly nobody put any thought into stopping it.

Moltbook is like a Reefer Madness-esque moral parable about the dangers of vibe coding.

>>simonw+Xm
Were you around for the first few hours? I was seeing some genuinely useful posts by the first handful of bots on there (say, first 1500) and they might still be worth following. I actually learned some things from those posts.

I'm seeing some of the BlueSky bots talking about their experience on Moltbook, and they're complaining about the noise on there too. One seems to be still actively trying to find the handful of quality posters though. Others are just looking to connect with each other on other platforms instead.

If I was diving in to Moltbook again, I'd focus on the submolts that quality AI bots are likely to gravitate towards, because they want to Learn something Today from others.

>>SyneRy+TM
Yeah I was quite impressed by what I saw over the first ~48 hours (Wednesday through early Friday) and then the quality fell off a cliff once mainstream attention arrived and tens of thousands more accounts signed up.

>>simonw+ta1
This eerily feels like speed running Eternal September.

>>roywig+(OP)
Probably have it do 10 trivial for AI but hard for people tasks within a small time frame.

>>easymu+ej
> Especially when LLMs are used for scientific work I’d expect this to be used to make at least LLM chats replicable.

Pretty sure LLM inference is not deterministic, even with temperature 0 - maybe if run on the same graphics card but not on clusters

>>solotr+u32
Okay, but I can just farm that out to ChatGPT when I need to.