Not even their ancestors at colonial times benefitted much from it: the industrial working class of Britain was in dire position despite Britain being a colonial Empire. That money and power went to the ruling classes and their middle class bootlickers.
It's very silly on the small scale. It's no less silly on the large scale, you are simply more accustomed to the cultural understanding the colonialism is not something you have any responsibility for.
I'm not talking about that happened thousands of years ago either. I'm talking about the conditions of the working and poor classes when Britain was a colonial superpower, like througout the 19th century. Hell, even post WWII most of Britain working classes were living very modestly, in wretched wretched council houses, and with low means.
Taking that example back to colonialism, it means probably returning stolen wealth and some portion of capital earned on the back of that stolen wealth.
It's a "my nation recently and currently systematically exploited people. I would like my nation to try to make those people whole" thing. It happens to be the case that many targets of exploitation were non-white, but the concern is the exploitation not the race of the exploited. We systematically exploited plenty of poor white people too, we should make them whole too.
Redlining in my city "ended" officially in 1968, but in practice it was probably another two decades before it really was removed from standard operation. And in my city, plenty of white people were considered undesirable and redlined. I guarantee you there are people directly impacted by this policy who are still alive, this isn't some far away past long since forgotten by time.
As an example, the city should be considering whether reduced interest rate loans, subsidized housing, rent freezes, or other benefits can be passed on to families directly impacted by redlining policies from the 60s, 70s, and 80s. Just because it's been 40 years doesn't mean the city should just give up trying to make people whole.