Computers Could Grade Essay Tests Better Than Profs 323
An anonymous reader writes "Robot essay graders could be the answer to grade inflation. New software being tested turns over the task of grading to computers — this article has an interactive demo of the software. One professor says the computer is far fairer than human graders, who get tired and become inconsistent, or play favorites."
Play favorites? I believe it (Score:5, Interesting)
As long as the robo-grader also includes a plagiarism check, I'd be okay with it. My husband is a professor and most of his failed papers are a result of TurnItIn.com catching outright plagiarism.
Re: (Score:3, Interesting)
Re:Play favorites? I believe it (Score:5, Insightful)
This is why most people don't take literary analysis seriously. There is a real human being who took the pain to write a 400-page long book. Presumably, he wanted to convey *something*. But apparently, we have to act as if the book came down from heaven and we can't try to discover what the author wanted to say?
The worst manifestation of this is when some literary theorists seem to argue that *even the author* cannot interpret what he wrote better than anyone else. He's just another reader!
This sounds ridiculous to me. Even if the author writes an essay saying "this is what I meant when I wrote this", we're supposed to ignore that and simply focus on the words of the work because this is all that matters in literary criticism?
Re: (Score:3)
This is why most people don't take literary analysis seriously. There is a real human being who took the pain to write a 400-page long book. Presumably, he wanted to convey *something*. But apparently, we have to act as if the book came down from heaven and we can't try to discover what the author wanted to say?
No you don't, but you can't say what Shakespeare thought, you can only say what you think Shakespeare might have thought. That's valid interpretation, and if you can back it up with the work of other critics then you're heading towards a supported academic position. If you claim to know what Shakespeare thought then everybody knows you are bullshitting because nobody does.
The worst manifestation of this is when some literary theorists seem to argue that *even the author* cannot interpret what he wrote better than anyone else. He's just another reader!
This sounds ridiculous to me. Even if the author writes an essay saying "this is what I meant when I wrote this", we're supposed to ignore that and simply focus on the words of the work because this is all that matters in literary criticism?
The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They'
Re: (Score:2)
The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They're not necessarily the same.
What was actually communicated is entirely subjective and will usually vary from person to person. What was intended is singular.
Re: (Score:3)
The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They're not necessarily the same.
What was actually communicated is entirely subjective and will usually vary from person to person. What was intended is singular.
What was intended might be singular but it rarely merits academic study and is inaccessible (even if the author is on record about it, how do we know that they were accurate and honest?)
The whole point about the humanities is that they are inherently subjective (there's a bit of a clue in the name), so learning to deal with subjectivity is an important part of a humanities course. What one person thinks is of some limited interest. The similarities and differences between what different people report findin
Literary Criticism (Score:3)
I wrote a response to this, but I think Slashdot ate it.
Has anyone done good empirical work on similarities and differences in perceptions of literature, according to cross-cultural, demographic, or other factors? The greatest weaknesses I have seen in the litcrit I've been exposed to have been the lack of empiricism, the lack of taste, and the lack of ability to write well (in fact, the propensity to write quite poorly, despite the use of jargon). But perhaps my exposure has not been broad enough.
Re: (Score:2)
It's hard to know, out of context, whether the grad student grading the paper was making an insightful criticism or a stupid one. Stupid grad student tricks abound, but stupid undergraduate tricks are even more common. It rather depends upon the work under analysis and what the student actually wrote.
With Shakespeare,, most of what we know about what he thought is from literary analysis of his plays and sonnets, supplemented with inferences drawn from general historical knowledge of the period, and what we
Re: (Score:2)
And that's even before you get into the question of whether Shakespeare was written by Shakespeare or by somebody else of the same name...
When as a grad student I did a close reading of an extract from King Lear, I was interested to note that the second folio version had a lot more of the sort of stuff that structuralists look for than the first folio version did, which at least raises the question of how much of Shakespeare's reputation is because of good editors.
Re: (Score:3)
No you don't, but you can't say what Shakespeare thought, you can only say what you think Shakespeare might have thought.
Uh... Duh? When someone says "Shakespeare might have intended to convey such and such," it takes a very special sort of pedant to assume the former and not the latter. I'm glad I studied a field that isn't full of such masturbatory bullshit.
Re:Play favorites? I believe it (Score:5, Insightful)
This is a bit of a strawman. In high school English, it was explained to us as: humans write literature, and sometimes they have something to say. This doesn't mean that they are the final word on what broader meaning their work has, but it does mean they have a deep insight into it. So no, don't ignore authors, but don't expect appeals to their authority to be viewed as anything but a fallacy in and of itself.
Or put another way: Read Robert Frost's "Stopping By Woods On A Snowy Evening" (http://www.iment.com/maida/poetry/frost.htm#stopping), (it's short). The dominant interpretation of this poem is that it is an allegory of old age and death. Frost, however, insisted that this poem was about nothing more than taking a ride through a wood on a snowy evening. Who's right? It's not an either/or. In literary analysis there are right interpretation*s* and wrong interpretations, but it's not like there's just one right answer.
Or at least that's what I remember from my last literary analysis class taken, which was in high school many years ago.
Re:Play favorites? I believe it (Score:5, Insightful)
I remember reading a quote from Tom Stoppard (playwright for Rosencrantz and Guildenstern are Dead, among others), in which he basically said that writing is like packing a bag that he later takes to the airport. The people at the airport opt to inspect his bag and they start finding all sorts of things that he didn't remember putting in the bag, but he can't deny that they are there.
There is an argument to be made that we should take art as it was intended, but oftentimes the benefit of something is other than what was intended (case in point: movies that are so bad they're good), so there shouldn't be a reason why we deny that line of thinking as well. That said, there should be a limit. Some people, particularly the sort of liberal arts folks we all love to lampoon, try to insert their own things into the bag, rather than finding things that were legitimately there in the first place. But if they're simply discovering additional, yet unintended, depth to a classic piece of literature that can help us appreciate it better? Yeah, I see no problem with that. It may be unintended, but that doesn't mean it's not there. Even so, we shouldn't ascribe more meaning to it than it's due.
Re: (Score:2)
Re: (Score:2)
Next time you run into Big William, please ask him what he meant by "to be or not to be" for me !
Re: (Score:2)
Taking that position, it makes it hard to understand why so many academics heap scorn on Wikipedia, which operates under a similar proscription: you can only write what you can source, even if what you wish to say is so obvious that no reputable source bothers to spell it out, in direct terms.
You can not state the insidiously obvious on Wikipedia. I once tried to add a footnote to a term from computer science that has lo
Re: (Score:2)
Ah yes, New Criticism, because Leavis and his idiotic ilk took a look at the downfall of logical positivism and said "if only our discipline could show such absurd hubris with even less justification". The second you make normative statements about criticism, is the second you look like a moron.
Before you mount a defence of your absurd position I have two words for you I consider highly appropriate given the context, assassination, Macbeth.
Re:Play favorites? I believe it (Score:5, Interesting)
Sometimes, things do work the way that it should.
Re: (Score:3)
I've seen worse, though not in Lit.
At Carolina, the Engineering program had a professor who, one semester, failed -everyone- ...except for the guy who was from the exact same part of India that he was.
Everyone there tromped down to the Dean's office and showed him the facts. They all got regraded, and the prof was not retained.
But worse than that was the Thermodynamics prof who graded entirely on the curve. As in, the Bell Curve.
The first test was six definitions and one problem involving steam. All but
Re: (Score:2)
At Carolina, the Engineering program had a professor who, one semester, failed -everyone- ...except for the guy who was from the exact same part of India that he was.
Do you mean South Carolina?
Re:Play favorites? I believe it (Score:4, Funny)
Perhaps it was simply a very long time ago, and he was referring to the Province of Carolina [wikipedia.org].
Not just favorites (Score:2)
I had a teacher that was reviled by just about the entire campus such was her utter lack of competence. I had initially thought it would be possible to go through her exams by bullshitting, but then I stumbled on something that made me understand some of it.
I had an appointment with her for reasons I do not recall and she was grading exams still. As I waited, she started discussing with another teacher, careless about me and many other students listening in. After the discussion ended, she decided to raise
Re: (Score:2)
Not really related, but I remember a chem test where there was a 4 part question, with each part supposedly graded independently from the others. Each part was worth 2 points.I got the first part wrong, and then you need to use your answer from part 1 in part 2, and so on. Having the first part wrong, naturally the numbers written for the others parts were wrong, even though the steps taken to acquire those numbers were completely correct. He docked points for each part that had the wrong number as the a
Re: (Score:2)
You learned an important lesson or two. Your TA was an asshole. Just because he's a teacher doesn't mean he's right. Cute chicks win (you did read Anthony and Cleopatra, didn't you?).
You wouldn't have learned that from a computer.
After school (Score:5, Interesting)
Re:After school (Score:5, Funny)
The lesson you probably learned is that buttkissing gets you further in today's society than delivering good work.
Who said school doesn't prepare you properly for your adult life?
Re:After school (Score:5, Interesting)
At the same time, if a customer tells you what features he wants and you keep not building it, are you surprised when he's unhappy with what you delivered?
Feedback is a valuable tool. What you do with said feedback is up to you.
Re: (Score:2)
one month out of the week
Strike that. Reverse it.
I swear it was the right way around in the preview.
Re: (Score:2)
Somehow a body that causes you pain and discomfort one month out of the week is worse than a body that causes you pain and discomfort 12 times a day without ever letting up
Wait...what?
Re: (Score:3)
Re: (Score:2)
Re:After school (Score:5, Interesting)
"That would be my concern also."
On the first paper we were allowed to re-write it and resubmit it, and rather than picking a new topic, I simply re-wrote the same paper from the opposite stance, parroting back the professor's (in my view entirely wrong) opinions. I even included some egregious BS about how I'd learned so much and realized how right he was. I worried it might be over-the-top with the sarcasm, but I couldn't help myself. Anyone without an ego problem would have seen through it, that a college student isn't likely to have a total change of heart and (in this instance) change from being basically a libertarian to being a socialist overnight because their professor was so brilliant that they showed them the error of their ways. I was a little scared he was going to notice and call me into his office for submitting a sarcastic paper.
I got an A. The rest of the class was a disgusting piece of cake. There was no reason to bother with hard work, insightful points, and original analysis. It wasn't even necessary to read the material (although I generally did for my own benefit.) I just typed whatever opinions the professor espoused in class, with fidelity that was borderline plagiarism, and it was an easy A every time.
Re: (Score:2)
Heh, I have a story like this from high school. Basically, I'd been put in the "scientist/non humanities" bucket from the start. Every time I tried to write something, whether it was a short story, a poem, or a critique, it got a crap grade. Points for figuring out set pieces like use of metaphor and that stuff, but "something was missing" if you get me. Or don't get me, because it was all BS.
So one day, I wrote a story about a guy kicking a ball around with his grandkid. Only this time, I smothered it with
Re: (Score:2)
Re: (Score:3)
I had the exact same problem with my Uni Maths classes. The Professor had some silly idea about what all the answers should be, and unless you parroted back the "correct" answers, he marked you as wrong. What a douche. Only a foll pretends that there are numbers beyond whole numbers.
The purpose of a test ... (Score:3)
I'd say the sarcasm was probably noticed but didn't cost any marks becuase it was used in a way that showed you were paying attention.
Sounds like a case of Poe's Law (Score:2)
Poe's Law relates to how parody of extremism can seem similar to actual extremism.
Accuracy (Score:5, Interesting)
I can't say if a computer is better than a human at marking, but in my engineering subjects, when my name was on the test papers I did not get very good grades (actually at least grade lower than expected). But as soon as all the students were given anonymous numbers the grades went up. Conclusion, the staff could no longer decide to give better grades to their pet students. So in theory, there could be many students who get better grades because there is no more favouritism.
Re: (Score:2)
I had to mark some student essays this year and, in common with the rest of the coursework, even though it wasn't anonymous, I had no idea who most of the students were. Most of them had been in my lectures, but even most of the ones who had asked questions or come and talked to me after the class never actually told my their names, so they were effectively anonymous. I had no idea who the guy who got 99% was until he asked in the lecture why he had a mark deducted.
As to whether a computer can be more
Re: (Score:2)
As to whether a computer can be more fair, if it can then you shouldn't be setting an essay. A computer can tell if you've listed a set of bullet points correctly, but it can't judge your understanding of the subject. For example, one of the titles my students could pick was 'Give five design patterns for concurrent programming and suggest when each would be appropriate'. Students got a reasonable mark if they showed me that they understood the materials I'd covered in the lecture. They got a really good mark if they showed me something I hadn't covered in the lecture and demonstrated that they understood it. How would you program a computer to make that call?
I agree. If you look at the example of computer grading in TFA, you'll see that it is in effect a set of bullet points.
http://chronicle.com/article/Can-Software-Make-the-Grade-/128505/ [chronicle.com] It's like a short-answer test with the numbers and paragraph marks deleted and run into a paragraph.
If you're going to use an essay-marking program like this, you might as well give short answer tests.
It's nice to get the date of birth and death in an essay, but today teachers are de-emphasizing facts that you can look up on
Re: (Score:2)
Interestingly, I had this discussion at work the other day. IMO, knowledge IS bullet points. Sounds weird, I know, but thinking back to every essay exam I've ever done, there always seemed to be certain things that had to be mentioned. How you mentioned them seemed to matter (subjective grading), but in the end, people who understand a subject are always gonna mention a certain set of ideas.
My colleague thought otherwise. Unfortunately, I don't have the bullet points of why.
Re: (Score:2)
I had to mark some student essays this year and, in common with the rest of the coursework, even though it wasn't anonymous, I had no idea who most of the students were. Most of them had been in my lectures, but even most of the ones who had asked questions or come and talked to me after the class never actually told my their names, so they were effectively anonymous. I had no idea who the guy who got 99% was until he asked in the lecture why he had a mark deducted.
As to whether a computer can be more fair, if it can then you shouldn't be setting an essay. A computer can tell if you've listed a set of bullet points correctly, but it can't judge your understanding of the subject. For example, one of the titles my students could pick was 'Give five design patterns for concurrent programming and suggest when each would be appropriate'. Students got a reasonable mark if they showed me that they understood the materials I'd covered in the lecture. They got a really good mark if they showed me something I hadn't covered in the lecture and demonstrated that they understood it. How would you program a computer to make that call?
I think subjectivity creeps in when it comes to the borderline cases. If a guy comes up with 5 patterns and manages to explain them like you would, you're inclined to think he gets it. But if a guy comes up with 5 sort-of-like-it explanations, containing some salient points, while omitting others, what do you give him? It can be hard to judge, especially as some facts are more important than others. Suppose there's one major fact and a couple of minor ones speaking in favor of a given DP, and the guy mentio
Re: (Score:2)
Of course. The module I taught was 50% essay and 50% programming. Of the 50% programming, about 5% was following a set of coding conventions and the rest was marked by a script. Some students got a few bonus marks for doing things beyond the requirements, and a few got some points for having code that was nearly right, just with some tiny bugs, but mostly the marking there was objective.
The essays were entirely subjective. The point of them was for the students to prove to me that they understood the c
Re: (Score:2)
Hey, I'm quite interested in high performance. Does your course have a website/notes/syllabus/anything? I'm one of those guys who learned by doing on the job, after uni. Got an MEng though.
Re: (Score:2)
We tried to use student numbers on papers at one point at a school I was at previously. Unfortunately it became pretty easy to tie a student number to a student, especially in a small class or lab.
You could come up with a unique ID for each submission, or if all submissions are done online you don't even need that, it's just a bin of assignments to grade with names removed (they still have a unique ID in the database it's just not exposed to markers).
Though I think with essays you get to know peoples style
Play favorites indeed (Score:5, Interesting)
Re: (Score:3)
My essay grades in college humanities courses were terrible until I started trying to figure out the political slant of my professor (or TA if the TA is the grader) and wrote papers supporting those views (and to be fair, those views weren't always left-leaning ones). I went from a C paper student to a low-A paper student in the blink of an eye.
That sounds like an excellent humanities lesson in itself.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
In Europe, most degrees are different from the US model in that they are less broad. So a math degree will be just math courses, no need to know who Shakespeare was. A literature grad won't need to know how to add two numbers together.
Re: (Score:2)
Re: (Score:3)
You don't have an obligation to agree with your teacher's ideas, but you do have an obligation to understand them.
Once you understand his ideas, you should be free to prove you understand him by repeating his ideas, then explain your reasons for disagreeing, and come to a conclusion different from his.
It doesn't always happen, but most teachers allow or encourage students to disagree with them, as long as they follow the academic conventions of supporting their arguments with evidence. (And as long as they
Playing favorites? ORLY? (Score:4, Interesting)
Consistency is a fair point, but playing favorites? Isn't this what anonymous marking codes/IDs are for? (Or at least, that's what happens in the majority of universities in the UK)
Re: (Score:2)
In the US with the exception of tests/exams, it is almost unheard of to not have your name directly on an essay. A few Universities or programs may do that, but the vast majority do not.
On standardized tests/exams the essays never have the authors name.
On professor generated exams, the grader usually can see the name, but professors setting up exams such that graders do not know the student's name, while rare, is not unheard of. If such an exam contains an essay, then the essay would be graded without knowi
Re: (Score:2)
It's a tough problem to solve. Ideally the instructor would be giving out rubrics when assigning the paper and sticking to it. At least that way folks have some meaningful idea as to what they need to do to get the grade they want. And can ask questions if they follow it and aren't given an appropriate grade.
The times that I get freaked out are when I've got an assignment due that's worth half my grade and the teacher hasn't bothered to spend any time explaining what exactly one needs to do to get a good gr
Great idea (Score:5, Interesting)
In addition, it would ideally be able to handle lab books. I remember grading micro-bio 201 lab books back in the 80's, and I was getting tired after the first 30. The second 30 was a pain. The last 30, well, we finished the grading at a pizza joint over beer. I suspect that was how grade inflation happens.
Re: (Score:3)
If it would improve your ability to appreciate the quality of my work, I'd be happy to include a joint in my lab book.
Graduate Record Exam (Score:3)
The GRE exam uses software to grade the essay portion for quite a while, along with a human grader. If these two scores different by a point or more, then it is forwarded to another human grader and the final score will be the average of the three entities.
That cuts the cost of running the exam, considering the cost of incurring an extra human grader.
It will soon pop up everywhere at university level, when the budget cuts are everywhere.
Re: (Score:2)
IBM system give a F for say toronto is in canada (Score:3)
And the IBM system give a F for saying Toronto is in canada. I say let the computer help but make it so there is no AUTO FAIL and make a real person review at least some flagged papers.
What's up with the mass media headlines? (Score:5, Interesting)
What's up with the mass media headlines? Reading the summary actually makes me dumber. It talks about "computers" like they are sentient and grades the tests instead. Having professors first strictly defining the rules, entering them into software and having a computer evaluate those rules is still "professors grading the essays". It's self evident that the grading is better if it's more strictly defined.
Wow, I can build a house faster with this hammer. Headline: Hammers Could Build Houses Faster Than Construction Workers (In Cyberspace)
Re: (Score:2)
Having professors first strictly defining the rules, entering them into software and having a computer evaluate those rules is still "professors grading the essays". It's self evident that the grading is better if it's more strictly defined.
True. If you read the example, you'll see that these "essays" are more like short-answer questions merged into a single paragraph. The program looks for a few pre-determined keywords. http://chronicle.com/article/Can-Software-Make-the-Grade-/128505/ [chronicle.com]
It might be more fun working a few dates and keywords into a paragraph than filling out short-answer questions, but the information is the same.
Fairer vs. Better? (Score:5, Insightful)
Even the (comparatively much easier) tasks of spelling and grammar checking result in a fairly steady stream of mistakes from computer systems. I can't exactly summon much optimism for the likely outcome of such a system trying to distinguish between a paper with a well supported thesis and a paper that contains some declarative statements, a few quotations, and the word "therefore" at intervals.
On the plus side, it should be pretty trivial to get the machines to do the same lousy job without the slightest consideration of the student's name/status/cuteness/willingness to flatter the professor; but what use is purely objective execution of lousy work?
Re: (Score:2)
I'm a bit puzzled as to how this scheme is supposed to work.
They gave an interactive example.
http://chronicle.com/article/Can-Software-Make-the-Grade-/128505/ [chronicle.com]
Re: (Score:2)
It could be these guys are using a similarly simplistic metric. The article doesn't really give any reason to believe their system is good, only that one particular professor who likes to teach large classes (up to 100
Re: (Score:2)
An essay graded by buzzword analysis is really just a fill-in-the-blanks where you have to supply your own connective grammar...
more classes need to move away from the written te (Score:2)
more classes need to move away from the written test and to a more hands on / maybe even no test class.
That fixes 2 things the people who just cram for the test and pass but I have little to no idea about the content or how to use it. As well as people who know what they are doing and are bad test takes / not that good at witting essays. Also cut's down on the people who pay for paper / essays witting services.
Re: (Score:2)
more classes need to move away from the written test and to a more hands on / maybe even no test class.
Good luck implementing that approach for journalism and creative writing courses! In some fields being able to write well is a vital skill. Even in sciences and engineering, the person who can communicate their ideas clearly and persuasively in writing will have a big edge over one who has similar technical skills but can't write well.
Re: (Score:2)
You've got that backwards. Written tests are of absolutely no value in Journalism or art. You might do a quiz covering something like ethics, but in terms of the actual practices you're not going to test that in any sort of useful way using a written test.
Too Late (Score:3)
As someone who never effected the curve or caught the affection of a teacher, I welcome our new digital grader overlords.
Writing for your audience just got harder (Score:2)
Let's be honest here, everyone had at least one teacher where it was obvious what you had to argue for or against to get a good grade. My German teacher was an ex-army officer. Take a wild guess what position you should take when the topic is the role of the armed forces in the history of the nation.
It was easy to get a good grade. Why? Because you knew, no matter how harebrained or outlandish your arguments were, if it was what he wanted to hear, a good grade was your reward. Simply and plainly. Once you l
Wasn't Aware (Score:3)
I'm a geology and earth science professor. When I give writing assignments, I'm usually more interested in the content than the mechanics. I'll tolerate a few spelling and grammar mistakes if the content of the essay or paper demonstrates that the student understands concepts presented in class and, even better, is THINKING about the implications.
For intro. writing classes, where grammar and structure are the point of the assignment, computerized grading is understandable; especially if your school has you teaching classes with more than 50 students (which is another issue entirely). But, in my experience at least, proficiency at writing is not always directly correlated with proficiency at class material.
Re:Wasn't Aware (Score:5, Informative)
I'm pretty sure no program is capable of this (and I've got a PhD in natural language processing). They might be able to check for a couple of easily scored factors, such as number of words, and consistency between paragraphs, but I'm pretty sure that there is no program that could distinguish between an essay and the same essay messed up to base reasoning on false assumptions. I think someone left out a pretty important assumption: such programs might be able to score fairer (meaning: with less bias!), provided the students did their best.
Re: (Score:3)
Take a look at SAGrader [sagrader.com] and see what it is doing. It's not grading "essays". It grades answers to narrowly focused essay questions. It's looking for key phrases. The student''s correct strategy is thus to repeat, exactly, the language of the textbook.
Re: (Score:2)
That's even worse, and it's not even essays (as the OP states). So the teacher has to provide the program with the "correct" anwers? That can be a pretty long list. It's probably going to be good at rating mediocrity...
I would believe that the grades are more objective (Score:2)
But the objectivity of the grades has nothing to do with the problem of grade inflation. Professors intent on inflating grades will simply reduce the weight of tests as part of the overall grade and count class participation, homework, etc. more /or/ add a flat number of points across the board to the results of the computer scored tests.
Grade inflation, after all, isn't simple bias. We're not speaking of professors grading up people (or views) that they like and grading down people (or views) that they dis
Doesn't help grade inflation (Score:2)
I'm not arguing that this is a good or bad idea, but it won't do anything to change grade inflation. In my experience (as a TA for a number of different classes), college professors look at the point totals at the end of the semester and determine the letter grade cutoffs by hand so that they have the grade distribution they want. I'm not saying they're going through and making sure specific students get a particular grade, just that they want, say 50% A's 30% B's and 20% C's and they'll put the cutoffs whe
Robotic Teachers - Robotic Students... (Score:2)
I was one of those graders (Score:5, Interesting)
First, for those who didn't read TFA, computers play only a small role on a handful of essays. Most of the article is in reference to having a 3rd party grade anonymized tests, rather than leaving it to the professor or TA. During college, I had a job as one of those graders.
We worked for five hours a day in the evening, though we could leave early and get the full pay if we finished all our papers. Most of the tests would be on general topics, but occasionally we'd get tests that required specific knowledge. In those cases, only qualified graders could review them, and we were given cheat sheets to make sure we didn't make factual mistakes. Essays were generally graded on a 1-5 scale (or a 0 if the essay was a blank page or similar). Each essay would be graded by two people, with a third breaking the tie in the event of a disagreement. However, we trained to be extremely consistent in the grading, so disagreements were rare and never more than a one point difference.
A few times a day, we would get fake essays intended to test our grading skills. For example, an essay that was supposed to be a perfect example of a 4 would be given to you with all the rest. If you gave it a 4, you get +1 point. Give it a 3 or 5, you get zero points. Give it a 2 or less, and you lose a point. If you accumulate a lot of points, you get a bonus up to 50% of your pay. If your total score goes too negative, you get fired.
It was a pretty good job, as crappy part-time "work your way through college" jobs go. The best part was whenever we got to grade essays by little kids. They were harder to score accurately -- it's hard to look past the abysmal handwriting and frequent misspellings. But they were frequently adorable and unintentionally hilarious.
It comes down to feelings (Score:2)
When machines are used to judge people's essays in subjective ways, either their state machines will be unemotional and miss much of the contextual meaning of writing, or they will be patterned after some designers' em
Are grades really meaningful? (Score:2)
My partner is just starting an MA teaching program, and she's been ranting a lot about the utter uselessness of grades and standardized testing. Apparently, there are decades of research establishing that standardized tests fail to measure anything but performance on standardized tests, and grades measure little besides conformism, self-discipline, and a lack of creativity. (And self-discipline is not always a good thing [alfiekohn.org] -- why are you working so hard at doing things you don't really believe are worth doing
Re: (Score:2)
So your ultimate solution is what now? I mean the sun causes cancer, but unfortunately for everyone, its essential to live. Why is this relevant? Well, if standardized testing and general schooling is so bad, what do you presume we replace it with? Rich people could get personal tutoring by experts in fields that they find meaningful. The rest of us should just learn the job our parents taught us because we get really really good at that one thing. Globalization would die because hell, if you can't generall
Re: (Score:2)
You're still working from the assumption that standardized testing is actually a meaningful test of whether someone has learned a subject. That's widely disputed. There's abundant evidence that tests and grades fail to predict anything other than future success on tests and future grades. They don't predict future professional success or future happiness.
Educators have lots of suggestions for alternative models of education; I've often heard something like my suggestion for teaching writing, for instance, a
Re: (Score:2)
standard tests are good because teachers on average are just average teachers - and that's pretty shitty, so if they got inherited tests, you at least as a student get to see some questions you're supposed to know how to answer/solve no matter how stupid the teacher is.
and every time you buy something built on the other side of the world, you're gauging if they managed to do it or not. if you're hiring them based on their chinese state school grades, you're a fucking idiot and your project is going to burn.
Re: (Score:2)
why are you working so hard at doing things you don't really believe are worth doing?
Because if everyone works hard at pretending to be firemen, cowboys, and ballerinas throughout their childhood, we'll end up with a lot of disillusioned firemen, cowboys, and ballerinas. Very few people end up doing things they really believe are worth doing. Even in IT, we may believe in our immediate goals (fix computers/design systems/plan networks), but few of us probably are 100% gung-ho about the company's products or goals. Thus the constant tests to make sure people would be good at following dis
Re: (Score:2)
worthless essays (Score:2)
Based on what I know about the current state of AI, the essays that computers can grade fairly are NOT worth writing.
Students and professors should try doing something useful... Something that pushes human intellectual boundaries and imagination, explores human emotions, discusses ethics and moral issues. These are not topics that today's artificial intelligence can handle.
From Degrading to De-Grading by Alife Kohn (Score:2)
http://www.alfiekohn.org/teaching/fdtd-g.htm [alfiekohn.org] ..."
"... The preceding three results should be enough to cause any conscientious educator to rethink the practice of giving students grades. But as they say on late-night TV commercials, Wait -- there's more.
Key points:
1. Grades tend to reduce students' interest in the learning itself.
2. Grades tend to reduce students' preference for challenging tasks.
3. Grades tend to reduce the quality of students' thinking.
4. Grades aren't valid, reliable, or objective.
5.
Re: (Score:2)
Re: (Score:2)
Education has at least three aspects, in decreasing order of importance:
* Personal growth in a variety of ways (including spiritual);
* Learning what you need to know to be a good citizen participating in political life (including voting);
* Preparation for doing specific useful vocational work.
Modern schooling has so degraded the notion of education that most people think it is mainly about the third item, and that is the example you drew from. There is also a fourth aspect in practice of schooling that has
Re: (Score:2)
Short summary: Alfie Kohn's a jackass. Picking on just one point (6) : just how does he intend to evaluate the student's progress, capability, and absorption of knowledge?
Grades are like the Force (TM): there's a light side and a dark side, and you need to know which is which.
FWIW, I found great pride and satisfaction in knowing I successfully solved all the (physics or math) problems on a test. I also felt pride and satis. when a Humanities prof. gave me a high grade on a paper AND included comments
Re: (Score:2)
This is too simplistic. (Score:3)
what about staff evaluation schemes?? (Score:2)
Aren't they a culprit too in grade inflation debacle ???
I was a TA in a far east university in an Engineering department. Generally I consider my self a tough marker, as I expect students to arrive at answers with right logical reasoning. Having said that, I usually had a partial blind eye for students who has genuine drive towards studies -- post grad research types --, because their future shouldn't be eclipsed by a one bad grade. Also I highly control the grade distribution, such that only 5-10% of the c
In defense of (some) professors. (Score:3)
I can see how in some cases the computer would do a better job than a professor. In particular, ones that could not care less about teaching. I'm in Physics, and in one grad course there was an essay on an exam that I got a zero on. When I looked at the solutions, it appeared that the essay on the key was actually my essay with a few slight modifications. Two sentences of the short paragraph were my words exactly. When I brought this to the professor (who was also my advisor), he (a) couldn't remember my name (b) wouldn't even look at the exam (c) wouldn't discuss the answer and deferred everything to his grader, who was another grad student. The grader had better things to do and just handed my exam back to me and said, "that's what you deserve." This same professor, it should be said, makes psychotic Wikipedia self-edits about how his work "reconciles quantum mechanics with the Christian faith", rarely talks to other groups about his research (once one of his students came to me to ask a question about a problem he'd been working on for months--within minutes I identified it as being identical to a well-known NP-hard problem), and frequently "dumps" RAs he doesn't like by simply ending all communication with them.
My point is, the professors and TAs that grade unfairly don't do so because they can't. They do because they don't care. When I graded essays, I had a list of things I wanted to see in a correct answer and how many points they were worth, and a list of things that I would always take off points for. Every essay had a column of numbers next to it and a copy of my rubric so that any student could see exactly what they got points for and what they may have been penalized for. Out of classes of over a hundred students, I rarely received any complaints except for students who were on the border of failing and were desperate for one or two points. While sometimes grading essays felt like a simple application of a regular expression, searching for the gems of knowledge, equally as important was the logic that led to that conclusion. Correct answers obtained through incorrect application of concepts weren't worth any points at all, and it would be difficult for a program to match that with any regular expression.
I guess experience with bad professors did teach me one thing--despite having no passion for teaching myself, I would always treat my students like people and do my best to ensure that they got the best education possible for their tuition.
Re:Playing favorites (Score:4, Funny)
Not who, what [oldcomputers.net].
Re:Playing favorites (Score:5, Insightful)
The best student. Duh.
Re: (Score:2)
Re: (Score:3)
As a grad student. My supervisor isn't allowed to ask me to grade work for him, or prepare lecture material for him. Some of them do that, but we have a union that allows us to push back against it. I am paid as a teaching assistant as well (which is not guaranteed for all grad students), for that I am to work no more than 140 hours in a semester, and supposed to be roughly 10 hours per week. Part of being an instructor (which I have also done) is setting assignments that can be graded in the hours you
Re: (Score:3)
Don't be suprised. I have seen PhD students in EE that does not know Ohms law. Chances are in both cases, we are encountering an international student. Make me wonder where they bought their bachelors from.