While I also agree with you on that, there are also prompts that make them not act like a person at all, and prompts can be write-once-use-many which lessens the impact of that.
This is why I tend to lead with the "quality of response" argument rather than the "user's own mind" argument.