It wasn't justified though. The humorless hordes who complained not only could not take a joke, but also were irate at everyone else not being similarly offended.
Perhaps his set isn't to the comedic tastes of everyone, but there's an easy solution to that: don't watch it if you don't like it.