To understand much of our language, Gnorts would have to already be aware that our words and symbols gain meaning from how they're used, and you couldn't, for instance, determine that a swastika is offensive (in the west) by its shape alone.
In this case, the term "colored people" gained racist connotations from its history of being used for discrimination and segregation - and avoiding it for that reason is the primary principle at play. There's also the secondary/less universal principle of preferring "person-first language".
It only works because we're in a society of judging people the moment we see them. Mimicking the language of "bad people" will get that association. I don't think we'll ever truly "fix" that.