The remnants of colonialism continue to produce winners and losers economically, with the winners stuck in local maxima where they extract value from the people, but the people themselves see only marginal benefit, and development is stuck at a snail's pace.
As with seemingly everything in life, the incentives for the different players really don't line up. Consumers lose, producers lose, and only a select few middlemen win anything at all.
The reason New York City is the biggest city in the US is because when the Erie Canal was built, the agricultural riches of the Midwest had a route to world markets. Where you have a major seaport, you also need major banks and major insurance companies to smooth out the financial needs of traders and shippers, providing the funds right away back to the farmers, instead of them waiting till the voyages were complete. (without the Erie Canal, New Orleans would have become the largest city in the US)
Yes, there is a lot of money in trading, banking, etc. At every step of the transaction pyramid, a %age is added to the price, and the %age fees charged on that go up accordingly. But that measures the true value of the product at each stage; if you have a cheaper way of getting the same product to the same stage cheaper, the (supposed) riches will be yours.
The socialist instinct ("anybody getting rich must be cheating") unfortunately obscures the real problem ("monopolists and cartels controlling supply and setting prices are the true enemies of the people") which hinders solving it; by putting capitalism in your gunsights, you make enemies out of natural allies.
It is not pleasant to think about it in these terms; but it does seem like some of the greatest improvements in general human welfare have their roots in relatively ungenerous undertakings by methodical, reasonable, self-interested actors. The Romans roads and the Pax Romana, and the profound legacy of Roman law, were not the result of a benevolent desire to help everyone in the world and save them from evil.
Can you provide some examples and resources to back up this hard truth?
Another way to get an intuition for it is the prevalence of various conveniencies of life. For example, I believe about two thirds of households in the world have refrigerators. About 12% of the world's households have cars. These and many other material benefits are possessed by regular people today but would have been out of reach 400 years ago to any member of the European nobility or, as Adam Smith puts it, "...an African king, the master of the lives and liberties of ten thousand naked savages.". (In his time, Smith was commenting on how remarkable it was that an English workman could have pots and pans made of metal and a window made of glass.)