I think that a large part of the issue is that ICE is violating human rights on a massive scale. Where they might be executing some laws, they're violating others.
And just a reminder, "just following orders" brings up some scary historical context that a lot of people don't want to help recreate.