We advertise the job as on-site only, and because of that the applications self-select for those that want in-office work. It's made our interviews more focused on technical ability.
I think this is a better equilibrium overall. Those on either side of the remote/on-site preference can find the right respective jobs and work cultures.