Study: Alphabetical Order of Surnames May Affect Grading (umich.edu) 72
AmiMoJo writes: Knowing your ABCs is essential to academic success, but having a last name starting with A, B or C might also help make the grade. An analysis by University of Michigan researchers of more than 30 million grading records from U-M finds students with alphabetically lower-ranked names receive lower grades. This is due to sequential grading biases and the default order of students' submissions in Canvas -- the most widely used online learning management system -- which is based on alphabetical rank of their surnames.
What's more, the researchers found, those alphabetically disadvantaged students receive comments that are notably more negative and less polite, and exhibit lower grading quality measured by post-grade complaints from students.
What's more, the researchers found, those alphabetically disadvantaged students receive comments that are notably more negative and less polite, and exhibit lower grading quality measured by post-grade complaints from students.
Well, that's just spiffy (Score:5, Funny)
That explains a lot of things from my past.
Re: (Score:2, Interesting)
Re: (Score:3, Insightful)
AI usually makes bias worse.
Re: (Score:3)
AI: Student's name is Aaronson, therefore correlating with grades on the internet by similarly named people means I must assign an A. Student Xi also gets a grade of X because that tickles my neural feedback loops.
Re: (Score:1)
AI usually makes bias worse.
AI makes artificial bias worse because it has less real bias.
For instance, an AI system used in New York recommended that more blacks than whites be denied bail.
That is artificial bias because blacks are less likely to show up for trial, so if that's the criteria, it isn't "real" bias to deny bail to more of them. It might not be "fair" or the right thing to do, but it isn't "bias". It's a recognition of reality.
Re: Well, that's just spiffy (Score:2)
Re: (Score:3)
You don't see an issue with looking at someone's skin colour and deciding that they are more of a flight risk?
Re: (Score:1)
Re: (Score:3, Insightful)
He became president because the competition was substantially worse. The competition also claimed he was smart because he had a very smart uncle, so apparently the competition also slept through the class on genetics.
Re: (Score:1, Offtopic)
It also could explain how the current president of USA has managed to ever become president given his limited aptitudes.
Yes, quite. Especially after his nonsensical remark about magnets [businessinsider.com]:
"Now all I know about magnets is this: Give me a glass of water, let me drop it on the magnets, that's the end of the magnets."
And let us not forget this rousing comment when he was at the Gettysburg battlefield [newsweek.com]:
"Gettysburg, wow—I go to Gettysburg, Pennsylvania, to look and to watch," he said. "And the statement of Robert E. Lee, who's no longer in favor—did you ever notice it? He's no longer in favor. 'Never fight uphill, me boys, never fight uphill.' They were fighting uphill, he said, 'Wow, that was a big mistake,' he lost his big general. 'Never fight uphill, me boys,' but it was too late,"
With this kind of aptitude, it must only be because of his name.
Re: (Score:2)
Or at least gives you an excuse.
Re: (Score:2)
Re: (Score:3)
When I graded, I did the opposite. Earlier tests or homework I looked at had the most scrutiny, then near the end of the pile I was tired and went more quickly. For me, more scrutiny meant lower grade, less scrutiny meant it looked just fine move on the the next. I don't think I graded in alphabetical order, just the order of the stack of work to look at. I never gave a final grade though, that was up to the prof.
Re: (Score:3)
The old "toss it down a staircase and the stair it falls on is the grade" trick doesn't care much about last name.
I have to agree with you though. The first few assignments I graded I tended to have high expectations. By the end those were gone.
Re: (Score:3)
I don't think I graded in alphabetical order, just the order of the stack of work to look at.
So the assignments on the top of the stack, which were the last to be turned in, received the lowest grades.
That doesn't seem totally unfair.
Re: (Score:2)
That presumes there's a correlation between when it was handed in and the student deserving scrutiny.
Any paper could end up on top for any reason - perhaps the teacher tells students to put them in a pile as they leave class - so the one on top is the one that left class last. Which may be a student who was asking a question after class. Or maybe they had an accident p
Re: (Score:1)
My high school English teacher eventually (end of senior year) confided to me that he had been in the habit of grading my papers last, so he could have at least one good paper to look forward to and finish on a positive note. (He liked my writing style; not everyone does, but he did. My papers always got good grades from him.) My surname starts with E, FWIW.
My point is, your grade isn't mostly determined by your p
Re: (Score:3)
In other news, you're more likely to receive a longer prison sentence if the judge hasn't had lunch yet.
Re: (Score:2)
Hey Canvas - Randomize the presentation to grader (Score:2)
There you go.
Re: (Score:1)
Randomness may not be required, but at least varying "who starts first" and maybe the "direction" (alphabetical or reverse-alphabetical) would add some fairness.
This assumes the phenomenon is real and not a one-off that will fail in a reproducibility study.
If this isn't real, it's not limited to Canvas - even in my grandparent's day some teachers sorted students work before grading.
If this IS real - major typo fix (Score:1)
If this phenomenon IS real, it's not limited to Canvas - even in my grandparent's day some teachers sorted students work before grading.
Re: (Score:2)
Canvas also has a setting to hide student names from people who are grading which can be enabled per-assignment and I think even now per-course (been a long time since I've done direct support and extreme use, but I still have admin powers/rights/etc as well as teach a course with it)
Re: (Score:2)
My guess is that the bias is due to the first artifacts to be graded have no baseline, and so they are graded the most generously. The more you grade, the more you see what's better and what's worse. What was a good paper if you saw it first might descend to the middle of the pile when you read nuances that others have introduced.
So randomizing the order would help spread around who goes early. But hiding the names without randomization still means the first papers get graded easier - and the same people st
Significant, but not a big difference (Score:4, Informative)
Of course the actual numbers are hidden deep within the article:
Their research uncovered a clear pattern of a decline in grading quality as graders evaluate more assignments. Wang said students whose surnames start with A, B, C, D or E received a 0.3-point higher grade out of 100 possible points than compared with when they were graded randomly. Likewise, students with later-in-the-alphabet surnames received a 0.3-point lower grade — creating a 0.6-point gap.
The difference, while big enough to be impossible to be a fluke, is also extremely small. 0.3 out of 100 possible points doesn't seem like something that could elevate George W. Bush from an idiot to a genius. Probably simply family connexions in that case.
Re: (Score:2, Troll)
Reminds me of that study about how kids born in january were more likely to follow a sports career.
Why? Because the kids born in january will have far more time to grow than the kid born in september or october before their first class. They will be bigger than their classmates, and they will perform better in sports class, receiving more praise and encouragements. And when it's time to choose a career, they will feel more confident in one that involves physical prowess.
It's really interesting how the tinie
Re: (Score:2, Funny)
Actually, the difference in being born earlier and thus being the oldest in class in terms of impact on eventual academic performance is far higher than this alphabetic difference.
The alphabetic difference is in fact so small, that one may argue that people with certain family names are actually simply less intelligent. This number can easily be achieved with a couple of people with say the name Quayle simply being very stupid because not many name start with a Q.
Called on first == more alert (Score:1)
Does a lifetime of being asked to line up first, being called by name first lead to better attentiveness?
Re: (Score:2)
There have been proposals to have two groups of students per year, to make the difference between the youngest and oldest 6 months instead of 12.
It seems to be too difficult to administer though.
Re:Significant, but not a big difference (Score:4, Insightful)
Of course the actual numbers are hidden deep within the article:
Their research uncovered a clear pattern of a decline in grading quality as graders evaluate more assignments. Wang said students whose surnames start with A, B, C, D or E received a 0.3-point higher grade out of 100 possible points than compared with when they were graded randomly. Likewise, students with later-in-the-alphabet surnames received a 0.3-point lower grade — creating a 0.6-point gap.
The difference, while big enough to be impossible to be a fluke, is also extremely small. 0.3 out of 100 possible points doesn't seem like something that could elevate George W. Bush from an idiot to a genius. Probably simply family connexions in that case.
Well, sort of. If you're the statistically average person, then it doesn't matter. However, if you're the unlucky person in the statistical distribution, then it matters more.
What's likely is that some graders are affected and some graders less so. So, those students with the more affected graders will have greater effects on their grades.
I totally believe in this effect. As a former TA with a heavy grading workload (like 10 hours per week), I definitely graded the latter assignments and quizzes significantly differently than the first few. It's not that the grades were intentionally higher or lower, but the time spent on each assignment significantly dropped. And this was grading where the answers were either right or wrong. Imagine a tired grader for something much more nuanced, like grading essays.
Re: (Score:2)
It's not even about being bad or tired. I've done a fair bit of grading in my time.
Even if you're well rested and happy (you are not), you definitely change over the course of marking because you get used to the exam in some way. You start spotting patterns, and also of course update the internal mark scheme in your head.
It's easy of course if the student get the right answer (or writes nothing). It's those partial credit answers that are a bitch figuring out what the student did and crucially how many mark
Re: (Score:2)
It's more than just the scores though. The study mentions that comments are harsher too.
In any case, it's an easy fix. Just randomize the order.
Re: (Score:2)
In any case, it's an easy fix. Just randomize the order.
Yes... but that is still not entirely fair. I mean usually in the UK system, students are assigned a number, and that's the only identifying mark on the exam scripts, which does randomize the order, though now the bias is present it's just randomly assigned which isn't great.
The American system way over indexes on exams, apparently on the grounds that the more the better. Se also the obsession with standardised tests. Yes... standardised tests are good
Re: (Score:2)
I dislike exams in general, too much pressure put on one event. I had issues on some exam days. Coursework seems better, if it can be reasonably protected from having the parents do the work.
I read somewhere some years ago that it was the norm to appeal any grade that fell below expectation on an exam, and often re-marking the work would raise it. Is that true?
Re: (Score:2)
I dislike exams in general, too much pressure put on one event. I had issues on some exam days. Coursework seems better, if it can be reasonably protected from having the parents do the work.
Or at uni level, other kinds of cheating. But yes.
There is no perfect form of assessment. Exams are bad. Coursework is bad. It's all bad.
I can think of plenty of counter examples of people who are just crap at exams and scraped a lower second (on the strength of coursework) while being technically incredible, and also p
Re: (Score:2)
If it's a bias that can be (somewhat) alleviated by randomization, then MORE small tests would help rather than fewer large ones.
With more tests, you'd scatter the error potential over a bunch of relatively small exams, effectively rendering it moot. With fewer tests, bias on any one has a much greater impact on your standing overall.
(And probably needing a pseudorandom distribution rather than true random to ensure some unlucky kid isn't the victim of long-odds and always landing at the back of the pack.)
Re: (Score:2)
"In any case, it's an easy fix. Just randomize the order."
To quote Marvin Minsky, "Well, it has [biases], it's just that you don't know what they are."
Re: (Score:1)
But it's often enough to elevate a grade from a B+ to an A-.
Not Significant (Score:2)
The difference, while big enough to be impossible to be a fluke
Really? Why? There is no uncertainty given on the measurement and they only quote it to one significant figure which implies that the uncertainty is at least 0.1 making that 0.3 gap less than 3 sigma from zero. Scientifically speaking that's not clear evidence of anything and signficances less than 3 standard deviations disappear all the time due to missing systematic effects or sometimes are even just statistical fluctuations.
Re: (Score:2)
Depends where marks are turned into grades then it is enough to push you down to a lower grade or even fail which can have life-changing consequences.
I am surprised that this is being suggested as new. It has been known about for years in the UK which is why there is strict randomization of papers and also anonymous marking to try and even out these effects.
People aren't free to tell the truth. (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Thank you for balancing the parentheses. Cosmic order has been restored.
Re: (Score:2)
I do note that your name seems to be pretty far down the list.
Re: (Score:2)
Re: (Score:2)
The farther you go down the alphabet you go, the dumber people get. You just can't say it out loud today.
I dunno... to me, it's always seemed like you bottom out almost immediately and just stay down there the rest of the way.
Re: So your saying it's the SYSTEM! (Score:2)
Subjectively it make sense... (Score:2)
In grade 8 my friend and I intentionally had two of our female classmates submit junk lab reports, and when the reports came back, we got lower grades. When we confronted the teacher: “Mrs Patterson”, she tried desperately to talk her way out of it. The girls got higher marks
Re: (Score:2)
We have maybe 50% of the teachers we need in the US, because we have maybe 50% of the teaching positions, and are willing to spend maybe 25% of the amount necessary to hire enough teachers at reasonable wages.
Really we need to be building approximately 1 school for every school we have now and hiring teachers accordingly so we can shrink class sizes.
However the education system we have now was deliberately compromised to avoid the creation of an "educated proletariat" (plug those words into google with the
Re: (Score:3)
I don't understand your point, I'm not arguing against hiring more teachers, I'm simply pointing out the quality of the teachers we have isn't great.
I live in Ontario, Canada, and we also have a teacher shortage, but the teachers we do have, from my experience, aren't great, and unlike “typical” US teacher compensation, we make sure they're taken care of. The average salary of a teacher in primary school in our province is 92k/year, and you get
Re: (Score:2)
I don't understand your point, I'm not arguing against hiring more teachers, I'm simply pointing out the quality of the teachers we have isn't great.
All of these things are part of the same problem. The best teachers are also the best equipped to take a job elsewhere. If they can't afford to survive on what's being paid, and they have other opportunities, they take them. There is also high burnout from having to be a babysitter to so many ill-mannered children, who got that way by emulating their ill-mannered parents.
We need more teachers, but that just means the teachers we have, need to work a little harder.
You sound like men that women won't talk to talking about women. Do you even know any teachers? I do, and they are working as hard as they
Re: (Score:2)
One of the things my younger daughter got in trouble for this year, paraphrased: “Refusing to say a land-acknowledgement, and then asserting she had nothing to apologize or feel bad for.”. Her teacher is
As a teacher who uses Canvas I see this (Score:4, Informative)
This is part of the reason I like objective tests. However, objective tests can not easily assess higher levels of learning. A lot of attention os paid to Coista's levels of questioning.
That said, it is not too hard to work around the problem mentioned here. The way I go about it is that I am consciously aware of when I change the expectations and what I have changed them to. I will then make a comment on the work of the student where I made the change. It may be a simple "It's great to see you making such progress," or something similar. Now you know the reason for seemingly pointless comments on tests.
Now, when I am done, I can go back to the material before my comment and be sure to apply the revised expectations. That way the same expectations are applied to all students. It is not hard to do, but I suspect some teachers skip the rescoring. I know it was something that I had to figure out on my own. It is good to see this article. It gives me something to bring up at my next meeting where I am expected to share "something."
Re: (Score:2)
Not buying it... (Score:2)
So as a current student that is using canvas, my last name starts with O and yet I have an A in all of my classes right now. Maybe people need to whine less and study more. Stop trying to find excuses and put in the work already.
Re: (Score:3)
Its surprising that you are getting As with your level of understanding of a relativity simple concept. The study didn't say that people with Z can't get As. It just that there was a bias against them 0.6 out of 100 relative to people who's names started with As. It doesn't even mean that a person with a Z can't get 100, its just on average.
Nobody was whining either just observing a phenomenon and maybe stating that could be compensated for.
Re: (Score:2)
Issue Closed:
WORKS FOR ME
Zuckerberg (Score:1)
In the middle of the alphabet (Score:2)
In school, I used to be annoyed that my last name starts with "I" which puts me roughly in the middle of the alphabet. So when students were selected in alphabetical order, I was somewhere in the middle. And when the teacher would change things up and reverse the direction, I was *still* right there in the middle!
This research explains a lot!
True for me, but for a different reason (Score:2)
When I was in school, teachers has a book with the names and grades. They usually called us in order to give oral presentation. Those who came later had the advantage.
Interrater reliability... (Score:1)
Anonymous coward! (Score:2)
Ahh, at last we know why the anonymous comments are usually much better!
These days I hardly bother reading upvoted comments here. It's like reading "critics" reviews on RT...
Well... (Score:1)
Wow this might be an actual use for a computer! (Score:2)
We don't even need AI for that! OR an internet connection!