I don't think it's fair. But I also believe that people should run their businesses how they see fit, and that freedom of association is a human right.
I bet most businesses would make their stores accessible (within reason) since you need room in isles for carts, etc.