Police carry guns. Legal is not relevant.
The difference is "qualified immunity", not weaponry.
Cops ultimately aren't disciplined because there's no political will. And there's no consequences on the ground because they have more weapons, and importantly ready backup from other cops.
If you were in a gang, and carried a gun, you could indeed arbitrarily force your will on others for some time.