The current problem is that the people who want to live in them are priced out of the areas where they would have to go by the zoning rules they would want to change. Having a 20% lower cost per square foot doesn't make it more affordable to move there when the smallest available unit is required to have three times as many square feet. But since they can't afford to move there they can't vote to change the law there.
It's effectively a local law against poor people living there, which the poor people can't vote against because they don't live there. That seems bad.