This justification can be extended a bit to people in executive management roles at corporations, but for regular employees? You either got their salary value out of them when they worked for you or you overpaid them. I don't see additional societal value to a non-compete except in edge cases where an employee quits within a short period after hiring - staying just long enough to gain skills without generating a corresponding amount of value for the company.
Yeah, for direct/important enough competitors, I can see it. Senior enough management can't help but have knowledge of a lot of things that neither the public nor most rank-and-file employees don't--and act on it at a new place even if they're not sending confidential board meeting presentations around. On the other hand, execution ability and culture matter for a lot too.