zlacker

>>isp+(OP)
A real Orderbot has the menu items and prices as part of the chat context. So an attacker can just overwrite them.

During my Ekoparty presentation about prompt injections, I talked about Orderbot Item-On-Sale Injection: https://youtu.be/ADHAokjniE4?t=927

We will see these kind of attacks in real world applications more often going forward - and I'm sure some ambitious company will have a bot complete orders at one point.

>>wunder+ns
I would expect these bots will be calling an ordering backend API which will validate the price of the items and the total. Are you suggesting people will plug open ended APIs that allow the bots to charge any amount without validations?

I think the first step will be replacing frontends with these bots, so most of the business logic should still apply and this won't be a valid attack vector. Horrible UX tho, as the transaction will fail.