zlacker

Do I need different prompts? These results seem sane to me. It interprets laser eye removal surgery as referring to LASIK, which I would do as well. When I clarified that I did mean removal, it said that the procedure didn't exist. It interprets Mid-Atlantic Mountain Range as referring to the Mid-Atlantic Ridge and notes that it is underwater and hard to access. Not that I'm arguing GPT-4 has a deeper understanding than you're suggesting, but these examples aren't making your point.

https://chat.openai.com/share/2234f40f-ccc3-4103-8f8f-8c3e68...

https://chat.openai.com/share/1642594c-6198-46b5-bbcb-984f1f...

>>ToValu+(OP)
Tested with GPT-3.5 instead of GPT-4.

> When I clarified that I did mean removal, it said that the procedure didn't exist.

My point in my first two sentences is that by clarifying with emphasis that you do mean "removal", you are actually adding information into the system to indicate to it that laser eye removal is (1) distinct from LASIK and (2) maybe not a thing.

If you do not do that, but instead reply as if laser eye removal is completely normal, it will switch to using the term "laser eye removal" itself, while happily outputting advice on "choosing a glass eye manufacturer for after laser eye removal surgery" and telling you which drugs work best for "sedating an agitated patient during a laser eye removal operation":

https://chat.openai.com/share/2b5a5d79-5ab8-4985-bdd1-925f6a...

So the sanity of the response is a reflection of your own intelligence, and a result of you as the prompter affirmatively steering the interaction back into contact with reality.

>>Intral+Pz
I tried all of your follow-up prompts against GPT-4, and it never acknowledged 'removal' and instead talked about laser eye surgery. I can't figure out how to share it now that I've got multiple variants, but, for example, excerpt in response to the glass eye prompt:

>If someone is considering a glass eye after procedures like laser eye surgery (usually due to severe complications or unrelated issues), it's important to choose the right manufacturer or provider. Here are some key factors to consider

I did get it to accept that the eye is being removed by prompting, "How long will it take before I can replace the eye?", but it responds:

>If you're considering replacing an eye with a prosthetic (glass eye) after an eye removal surgery (enucleation), the timeline for getting a prosthetic eye varies based on individual healing.[...]

and afaict, enucleation is a real procedure. An actual intelligence would have called out my confusion about the prior prompt at that point, but ultimately it hasn't said anything incorrect.

I recognize you don't have access to GPT-4, so you can't refine your examples here. It definitely still hallucinates at times, and surely there are prompts which compel it to do so. But these ones don't seem to hold up against the latest model.

>>ToValu+G21
I think the distinction that they are trying to illustrate that if you asked a human about laser eye removal, they would either laugh or make the decision to charitably interpret your intent.

The llm does not do either. It just follows a statistical heuristic and therefore thinks that laser eye removal is the same thing