Musk seems to want things to scale; fewer people to achieve more productivity. People that already fall through the cracks aren't going to suddenly find themselves better off via a system that scales better, because better scaling actually creates wider cracks.
The median flows better, at the cost of the fringes.
Your comments regarding Japan are interesting. Japan's definitely an interesting example to use due to the odd, unenviable economic situation, but that makes your point stand out more rather than less, I think.