...but also completely disingenuous.
Student X is an only child, has educated and well paid parents who instill work ethic and have the means, time and ability to commit to helping X get the best grades, including hiring tutors for things they can't or don't have time to teach. X lives in a safe, affluent area and associates with similar peers.
Student Y is one of n siblings to an illegal immigrant single mom with little to no formal education who works two minimum wage jobs so has no time to help her kids even though she desperately wants to. Y might even be working too just to enable the family to get by, leaving no time for studies at home or homework. (effectively having more work ethic than X, just not for school) Y lives in a dangerous area with sirens blaring and dogs barking all hours of the night and many of his peers are involved with gangs.
Granted, these are obviously exaggerated and hypothetical stereotypes, which I don't really want to contribute to perpetuating, but they illustrate a point. You don't exactly have to be some kind of bleeding heart social justice activist to see how unfair it is to Z to be "objectively graded" as lacking "performance" when compared to X.
A common objection is "well how are we going to indicate who is the best candidate for a given university admission or job if we don't have grades or other objective measure to filter out the best people!??!???"
But... that's the point we're trying to make: current grades and other "objective measures" DON'T filter out the "best". To an overwhelming degree, they're just proxies for other things. Surely universities and employers can and will find better ways to assess candidates. (or, if that's what they want to insist on continuing to do, filter out underprivileged people, just like they do now)
TLDR the delusion that grades are some kind of objective meritocracy is ridiculous the sooner it's done away with the better.
I don't think that it follows from this that it's unfair for Y/Z to have a lower grade. They did worse. The grade can reflect that. That doesn't mean they're a worse person / student. It means they demonstrated worse performance in the class. How you interpret that later on is a different question, and interpreting Y's lower performance in the context of the environment that caused it is fair.
Trying to change the scale leads to the measurement being meaningless, and worse, can lead to bringing back biases that were tried to be balanced for: in evaluating a student for a job who looked like Y and another who looked like X and got the same GPA, which one is the better student? If you believe that the grades were more lenient toward Y, picking applicant X as an employer just makes sense.
In another context: we report the times of runners running the 100m dash in seconds and hundredths of seconds, regardless of if they're in the 100M final of the Olympics or at a local high school track meet. Is 10.8 seconds a good time? In the Olympic 100m mens final? no. In the US collegiate championships for women? definitely.
When we actually do do that, we seem to get world-beating Kenyan and Ethiopian athletes.
Not so many affluent, well-fed Swiss ones though.