The rich will ALWAYS get their piece of the pie, and once they've had their fill, we'll be left fighting for the crumbs and thanking them for their generosity.
AI won't solve world hunger, it will make millions of people jobless. It won't stop wars, it will be used as a tool for the elite to spread propaganda. The problems that plague society today are ones that technology (that has existed for decades) can fix but greed prevents it.
Well here comes AI to take those jobs. What happens, you think? Where do we go next? Do you imagine we'll all just sit idle and give out orders for the AI to fulfill? Recall: the system throws away parts that are not useful. And we're not better at orchestrating this system than we are at implementing it. Most people already struggle to handle the complexity of modern life. So they'll be thrown out.
Now think what happens with a society where most people are unemployed, unhappy and hungry, and businesses are mostly, not ENTIRELY mind you, but most self-sufficient machinery that does the thinking and does the footwork?
But even that doesn't describe the problem alone. It's more of an end game. Before this we'll see not-so-superior AI pollute our web, media, public space with quickly generated content, as actual artists and thinkers are displaced, unable to compete. Our culture will die first. And then, eventually, we'll start dying.
As I'm describing this, note I don't say this from place of fear. I don't fear this. I see it more as an obvious place for our civilization to go. We can't help it, because we don't decide where this civilization goes any more than your cells decide where you go, or any more than the atoms of your cells "decide" where the cell goes.
We're not in control. That's just evolution.
Say, when you're sick and you have cancer, those cells are part of you, but they harm you, so you cut them out, apply chemotherapy, and then if there's a prosthesis to substitute the organ you removed with a machine, you do it, and you don't think twice about it.
What makes you think our society as a whole is different? If humans are not good at what society needs, it cuts those people out, and replaces them with working machines. It's so plainly obvious. We pay lip service to human rights and the value of an individual, but clearly that's not what we end up doing. A politician is chasing money and power, and they don't mind starting wars to get them if they can. A business chases profit, so they don't mind automating away any employee they can. It's always been this way. So now that you can replace the human thinkers, businesses won't need human thinkers. And since there's nothing left humans are good at, society won't need humans.
And those which are carried in our pockets are no longer capable of being home brewed.