People knew slavery was wrong when slavery was happening. People knew child labor was wrong when child labor was happening. People knew segregation was wrong when segregation was happening. Those people were not rewarded by society.
This isn't a reliable method of determining morality.