Forgot your password?
typodupeerror
AI Education Science

Computers Could Grade Essay Tests Better Than Profs 323

Posted by timothy
from the mckittrick's-behind-this dept.
An anonymous reader writes "Robot essay graders could be the answer to grade inflation. New software being tested turns over the task of grading to computers — this article has an interactive demo of the software. One professor says the computer is far fairer than human graders, who get tired and become inconsistent, or play favorites."
This discussion has been archived. No new comments can be posted.

Computers Could Grade Essay Tests Better Than Profs

Comments Filter:
  • by sandytaru (1158959) on Sunday August 07, 2011 @12:38PM (#37014868) Journal
    I once got an F on a paper from a TA who wrote in the margins "How dare you try to say what Shakespeare was thinking!" Um, that's what literary analysis IS, to some extent. You try to place someone's written works within the context of their culture and society at large and reconstruct their thought processes and views on the world. But that TA was an asshole and had it out for me, and many of us complained about him bitterly for years afterward. The only person who got an A in that entire section was one cute girl.

    As long as the robo-grader also includes a plagiarism check, I'd be okay with it. My husband is a professor and most of his failed papers are a result of TurnItIn.com catching outright plagiarism.
  • After school (Score:5, Interesting)

    by digsbo (1292334) on Sunday August 07, 2011 @12:39PM (#37014870)
    I had a prof in literature who only graded well if you made your critical essay about sexual imagery. At one point I gave up trying to "be me" and went whole hog, way overboard, almost parodying the over sexualized essay. And I scored an "A" for the the first time. Lesson learned? Sometimes it's OK to tell the boss what he wants to hear and do it his way, as long as it doesn't cost you anything, and nobody gets hurt. And, of course, life's not fair.
  • Accuracy (Score:5, Interesting)

    by Wowsers (1151731) on Sunday August 07, 2011 @12:39PM (#37014880) Journal

    I can't say if a computer is better than a human at marking, but in my engineering subjects, when my name was on the test papers I did not get very good grades (actually at least grade lower than expected). But as soon as all the students were given anonymous numbers the grades went up. Conclusion, the staff could no longer decide to give better grades to their pet students. So in theory, there could be many students who get better grades because there is no more favouritism.

  • by hal2814 (725639) on Sunday August 07, 2011 @12:40PM (#37014886)
    My essay grades in college humanities courses were terrible until I started trying to figure out the political slant of my professor (or TA if the TA is the grader) and wrote papers supporting those views (and to be fair, those views weren't always left-leaning ones). I went from a C paper student to a low-A paper student in the blink of an eye.
  • by DavidR1991 (1047748) on Sunday August 07, 2011 @12:40PM (#37014888) Homepage

    Consistency is a fair point, but playing favorites? Isn't this what anonymous marking codes/IDs are for? (Or at least, that's what happens in the majority of universities in the UK)

  • Great idea (Score:5, Interesting)

    by WindBourne (631190) on Sunday August 07, 2011 @12:41PM (#37014894) Journal
    but it really needs to check for plagiarism. I saw a load of it up at Colorado State.

    In addition, it would ideally be able to handle lab books. I remember grading micro-bio 201 lab books back in the 80's, and I was getting tired after the first 30. The second 30 was a pain. The last 30, well, we finished the grading at a pizza joint over beer. I suspect that was how grade inflation happens.
  • by hal2814 (725639) on Sunday August 07, 2011 @12:48PM (#37014950)
    The purpose of literary analysis is to analyze literature, not the author writing it. It was an asshole way to put it, but the TA was correct. It doesn't matter what the author thought or even intended. The only thing that matters is what the author wrote and what we can analyze from that.
  • by WindBourne (631190) on Sunday August 07, 2011 @12:49PM (#37014956) Journal
    Back at NIU, I had a lit class in which the female prof on the tenure track nuked all 3 guys in the class. I mean we scored D and F. ALL of the women scored much higher. I got fed up with this and one of my dorm mates gave a paper of his that had earned a A+ from the head of the lit program. It got a D-. After the semester was done, we took all of my papers including the purposely plagiarized one and went to the head. Showed it to him. Apparently, a major investigation was done, and she was released after that. My grades were adjusted up to a B after the head had re-graded all of the men's paper (I gave the head the paper that I had done and let him grade it).

    Sometimes, things do work the way that it should.
  • by Co0Ps (1539395) on Sunday August 07, 2011 @12:50PM (#37014964)

    What's up with the mass media headlines? Reading the summary actually makes me dumber. It talks about "computers" like they are sentient and grades the tests instead. Having professors first strictly defining the rules, entering them into software and having a computer evaluate those rules is still "professors grading the essays". It's self evident that the grading is better if it's more strictly defined.

    Wow, I can build a house faster with this hammer. Headline: Hammers Could Build Houses Faster Than Construction Workers (In Cyberspace)

  • Re:After school (Score:5, Interesting)

    by furball (2853) on Sunday August 07, 2011 @01:15PM (#37015172) Journal

    The lesson you probably learned is that buttkissing gets you further in today's society than delivering good work.

    At the same time, if a customer tells you what features he wants and you keep not building it, are you surprised when he's unhappy with what you delivered?

    Feedback is a valuable tool. What you do with said feedback is up to you.

  • by artor3 (1344997) on Sunday August 07, 2011 @01:24PM (#37015256)

    First, for those who didn't read TFA, computers play only a small role on a handful of essays. Most of the article is in reference to having a 3rd party grade anonymized tests, rather than leaving it to the professor or TA. During college, I had a job as one of those graders.

    We worked for five hours a day in the evening, though we could leave early and get the full pay if we finished all our papers. Most of the tests would be on general topics, but occasionally we'd get tests that required specific knowledge. In those cases, only qualified graders could review them, and we were given cheat sheets to make sure we didn't make factual mistakes. Essays were generally graded on a 1-5 scale (or a 0 if the essay was a blank page or similar). Each essay would be graded by two people, with a third breaking the tie in the event of a disagreement. However, we trained to be extremely consistent in the grading, so disagreements were rare and never more than a one point difference.

    A few times a day, we would get fake essays intended to test our grading skills. For example, an essay that was supposed to be a perfect example of a 4 would be given to you with all the rest. If you gave it a 4, you get +1 point. Give it a 3 or 5, you get zero points. Give it a 2 or less, and you lose a point. If you accumulate a lot of points, you get a bonus up to 50% of your pay. If your total score goes too negative, you get fired.

    It was a pretty good job, as crappy part-time "work your way through college" jobs go. The best part was whenever we got to grade essays by little kids. They were harder to score accurately -- it's hard to look past the abysmal handwriting and frequent misspellings. But they were frequently adorable and unintentionally hilarious.

  • Re:After school (Score:5, Interesting)

    by Phat_Tony (661117) on Sunday August 07, 2011 @01:45PM (#37015436)
    Similar experience here. I got very good grades on my college papers. Later on I had a sociology class and got a bad grade on my first paper. I knew it was a much better paper than that. I talked to the TA and she tactfully went over some things I could improve that sounded mostly like BS she was trying to make up because she didn't know what she could tell me. After a pause I said "it is my concern that I can not get a good grade in this class without agreeing with the professor's opinions," and she replied:
    "That would be my concern also."

    On the first paper we were allowed to re-write it and resubmit it, and rather than picking a new topic, I simply re-wrote the same paper from the opposite stance, parroting back the professor's (in my view entirely wrong) opinions. I even included some egregious BS about how I'd learned so much and realized how right he was. I worried it might be over-the-top with the sarcasm, but I couldn't help myself. Anyone without an ego problem would have seen through it, that a college student isn't likely to have a total change of heart and (in this instance) change from being basically a libertarian to being a socialist overnight because their professor was so brilliant that they showed them the error of their ways. I was a little scared he was going to notice and call me into his office for submitting a sarcastic paper.

    I got an A. The rest of the class was a disgusting piece of cake. There was no reason to bother with hard work, insightful points, and original analysis. It wasn't even necessary to read the material (although I generally did for my own benefit.) I just typed whatever opinions the professor espoused in class, with fidelity that was borderline plagiarism, and it was an easy A every time.

Philogyny recapitulates erogeny; erogeny recapitulates philogyny.

Working...