Here are several real stories I dug into:
"My brick-and-mortar business wouldn't even exist without AI" --> meant they used Claude to help them search for lawyers in their local area and summarize permits they needed
"I'm now doing the work of 10 product managers" --> actually meant they create draft PRD's. Did not mention firing 10 PMs
"I launched an entire product line this weekend" --> meant they created a website with a sign up, and it shows them a single javascript page, no customers
"I wrote a novel while I made coffee this morning" --> used a ChatGPT agent to make a messy mediocre PDF
The last one I did it on was breathlessly touted as "I used [LLM] to do some advanced digital forensics!"
Dawg. The LLM grepped for a single keyword you gave it and then faffed about putting it into json several times before throwing it away and generating some markdown instead. When you told it the result was bad, it grepped for a second word and did the process again.
It looks impressive with all these json files and bash scripts flying by, but what it actually did was turn a single word grep into blog post markdown and you still had to help it.
Some of you have never been on enterprise software sales calls and it shows.
Hah—I'm struggling to decide whether everyone experiencing it would be a good thing in terms of inoculating people's minds, or a terrible thing in terms of what it says about a society where it happens.