Me forgetting/never having "recorded" what necklace the other person had during an important event is not at all similar to a statistical text-generation.
If they ask me the previous question I can retrospect/query my memory and tell 100% whether I know it or not - lossy compression aside. An LLM will just reply based on how likely a yes answer is with no regards to having that knowledge or not.