It's usually not black and white. Sometimes the safe way could simply be to reduce speed, change lanes pre-emptively, etc. And when an emergency situation does happen, reaction speed is key. Good judgment is doing things in a timely fashion and decisively. A distracted human would drop the ball on both fronts not realizing they are in danger and then doing the wrong thing or act too late or overreact.
That's how people get killed on roads. Early experience with self driving taxis seems to suggest that journeys are uneventful and passengers stop paying attention and leave the driving to the car. So, yes, they quickly accept that the car is driving the car just fine.