I think this is one of the two culture clashes happening with police versus civilians right now.
After the protests in the wake of the Michael Brown killing, there were articles written[1][2] about how there are a ton (many dozens) of tiny towns around Saint Louis each only a few city blocks large that prey on their citizens like parasites. Middle class people have largely moved out of these towns so they avoid this problem.
I don't think "defund the police" is a particularly good slogan and this is likely the reason other comfortably middle class families are likely to reject that calling.
But I also think that those who are comfortably middle class and ignoring the very real struggles of the lower class at the hands of the police and their governments have a duty to help lift their fellow Americans up and at least learn about the problems they face, if not do something about them. If we don't, we are no better than the pre Civil War city dwellers of the North comfortably ignoring the plight of the slaves and the indentured servants in the agrarian south.
[1] https://www.npr.org/2014/08/25/343143937/in-ferguson-court-f...
[2] https://www.washingtonpost.com/news/morning-mix/wp/2015/03/0...
The trouble as I see it is that police broadly have two functions: to respond to emergencies where violence is likely and to suppress or tax the grey and black economy (huh. just noticed the ufnordunate linguistics of that). The 'responding to emergencies' function will of course still need to be performed, but it's fairly reliable that the poorer a neighbourhood is the less likely the residents are to voluntarily call 911 if an emergency does happen. In truly poor places the 'taxing grey and black markets' part of police work completely overshadows the good done by the 'responding to emergencies' part.
That is understandably very difficult for wealthy people to imagine, because they have very few examples of grey or black markets in their own lives, let alone instances where they have no other options. The thought that most people in any area would rather have enterprising teenagers sell bootleg cigarettes without hassle isn't even repugnant to them, it's just foreign.
The inescapable conclusion to withdrawing the police is: 1 - response times will go up drastically, 2 - security will deteriorate, 3 - wealthier and middle class folks will stop visiting/living in those areas because of #1 and #2, 4 - the tax base will evaporate, 5 - the infrastructure, schools, and all other communal areas will deteriorate, 6 - the class divide will become a gaping chasm between the poor who are relegated to the slums and everyone else. We don't have to try this again. We know that this is what happens from our own history in the 20th century!