Food, gambling, etc. are all backed by hordes of brilliant well paid people trying to get you to ruin your life so they make money. On the other side is just regular people like us stressed out trying to survive.
This isn't some "freedom" issue, it's an incredibly huge power asymmetry and I think "we the people" need protection from these forces
The line at the gas station of people buying scratchoffs and lottery tickets is proof.
The part we likely need protection from is the marketing.
If murder was legal we'd have a lot more of it. We still have them despite it being a crime, but nobody would ever suggest making it legal because some people do it anyway.