Not exactly true. Given an image, you can find the closest point in the latent space that image corresponds to. It is totally feasible to do this with every image in the training set, and if that point in the latent space is too close to the training image, just add it to a set of "disallowed" latent points. This wouldn't fly for local generation, as the process would take a long time and generate a multi gigabyte (maybe even terabyte) "disallowed" database, but for online image generators it's not insane.