To understand much of our language, Gnorts would have to already be aware that our words and symbols gain meaning from how they're used, and you couldn't, for instance, determine that a swastika is offensive (in the west) by its shape alone.
In this case, the term "colored people" gained racist connotations from its history of being used for discrimination and segregation - and avoiding it for that reason is the primary principle at play. There's also the secondary/less universal principle of preferring "person-first language".
In terms of racism, its different but the same mechanism. Being compared to a minority race is not an insult (to most people). Its the fact, that racist people will use the word with vitriol. Racists and those they argue with will use the term in their arguments and gradually the use of the term will gain the conotation of a racist person. Hence, Negro -> Colored -> Person of Color -> <the next thing when PoC becomes racist>