Oh my god, I'm glad someone else noticed this because it was driving me nuts. Deeply troubling that their case hinges on such a fundamental misunderstanding of the technology.
I bet a proper analysis of that toy experiment would conclude that none of the original data points are perfectly recovered: Only the underlying distribution / manifold is recovered, which really doesn't lend well to their argument at all.