The condition of being descendants of slaves, or people who faced other forms of official discrimination cited in the prevention of intergenerational wealth such as redlining, blockbusting or unfavorable treatment in the GI bill, etc., is ultimately an individual situation for each individual affected.
The idea that you can dismiss that as not an individual hardship -- though it kind of is for those impacted -- strikes me as pretty much a word game, nothing more. Not unlike the word games American laws started to use when they could no longer punish people de jure for their race.
Poverty is disproportionately in minorities' court due to historical discrimination based on skin color. That doesn't mean that there aren't poor white people. But it does mean that actions targeting minorities end up overlapping a lot with poverty.
The main reason it's relatively easy to target skin color is, well, visuals. Another can of worms in and of itself, but for the most part it's pretty easy to look at a certain minorities and pin them as such.Meanwhile financials are private and it's not like every millionaire is driving a fancy car with a suit and tie.