Even if "only" 10% of elite kids go on to become elite adults, 10% is orders of magnitude larger than the base percentage of adults who are elite athletes, musicians, etc. This doesn't sound "uncorrelated" to me so much as "not as strongly correlated as one might expect."
And describing something that happens 10% of the time as "rare" sounds a bit weird, like referring to left-handedness (also about 1 in 10) as rare.
A child prodigy in tennis may find that their body didn't grow in such a way to be a pro as an adult. If your opponents are taller, stronger, have better VO2Max, etc. than you as an adult, it doesn't matter how good you were as a child--they're going to beat you as an adult.
Chess, of course, now provides the stark reverse contrast. If you weren't a child prodigy in chess, you simply will not excel against the competition as an adult.
You can be the #1 rated player up to your last year of high school but if you don't hit the growth spurt required for your position your career will take a completely different turn. Conversely, it is the only sport I am aware of where you have people playing at the highest level who picked up their first basketball at 16