MIT's SAT Math Error

theodp writes "The Wall Street Journal reports that for years now, MIT wasn't properly calculating the average freshmen SAT scores (reg.) used to determine U.S. News & World Report's influential annual rankings. In response to an inquiry made by The Tech regarding the school's recent drop in the rankings, MIT revealed that in past years it had excluded the test scores of foreign students as well as those who fared better on the ACT than the SAT, both violations of the U.S. News rules. MIT's reported first-quartile SAT verbal and math scores for the 2006 incoming class totaled 1380, a drop of 50 points from 2005."

1220 in 1989

[Saint Stephen]

You can't compare any scores because it's all been rebased to be meaningless.

Back then, a 1400 really meant something, and a "perfect" score was a one or two person thing.

Re:1220 in 1989

[JoelKatz]

That's correct, the scores are normalized so that the distributions are the same. This means you *can't* compare scores across years. If you did, you would find that, amazingly, the distributions were the same. But have the students stayed the same? Nope. Have the questions stayed the same? No again.

If you google around, you'll see articles about how "national SAT scores fell for the second year in a row" or some nonsense like that. There are ways you can sensibly compare SAT scores across years, but you cannot compare averages over a significant fraction of the testing pool.

Re:Oops!

[Rakishi]

US colleges use a whole lot more than the SATs to determined admission, essays and extra-curriculars and grades and so on. In some ways the fact that you can study for the SAT does make it a better measure, work ethics and the ability to study are important for life and college.

Actually the US college system relies amazingly little on standardized tests in comparison to many other nations. In many countries there is a set of tests which pretty much are the only measure and the only chance you get. If you do badly or the computer system fucks up you're screwed.

Re:Oops!

[arth1]

He wasn't bitching about it -- he was stating a fact: that he was doing worse than others, and did not blame it on anyone else. The only one here bitching is you.

Re:If I could do it all over again...

[ResidntGeek]

It sounds like you don't have a chemistry or nuclear physics degree.

That's true. I'm currently in my second year studying chemistry. I didn't necessarily mean the degrees wouldn't be useless professionally; I'm very aware that the world is not a meritocracy. I do think that a good education is its own reward, and you have to study a real subject in the arts/sciences for one of those.

By the way, I had been under the impression that engineering degrees were generally for people who wanted to make money (in a normal-ish job) after graduation, while sciences were for people who either wanted to be, or accepted the risk of being in academia for life. Is that not the case?

Re:1220 in 1989

[NMerriam]

I was under the impression that SAT scores were normalized, so the distribution was the same between years. Am I wrong? Anyone have any sources?

Yes and no, one problem is that now they normalize the test TOO often, due to the fact that students weren't scoring well (average SAT score fell to about 930-950 or so by the early 1990s). They added essays and some other stuff which arguably added more subjectivity to the grading, and they did a BIG recalibration in 1994 that basically gave everyone an extra hundred points (don't they allow calculators now, too?). So any test scores from 1994 or later are considered meaningless as anything other than an indication of how you did on the SAT compared to the other students that exact same year.

Before 1994, the SAT correlated closely with IQ and could generally be compared (roughly) across years because it hadn't changed much in decades (precisely the complaint that led it to being redesigned). For example, MENSA doesn't accept [mensa.org] SAT scores after 1994 as indication of intelligence.

Re:If I could do it all over again...

[mr_mischief]

Well, English and philosophy are two of the recommended undergrad degrees taken in conjunction with pre-law to gain entry into law schools. So the Bachelor's degree by itself may not be spectacular, but it can set one apart from all those political science majors. History might also be a good choice. See University of Missouri St. Louis Political Science department's information on studying Law for just one program that mentions English and philosophy both as options.

Also, consider that many state government positions have a prerequisite of any Bachelor's degree from any accredited college. In Illinois, for example, many decent jobs with good benefits can plausibly be had with a degree in Liberal Arts or Medieval Literature, although you might be up against candidates who might have studied something more directly relevant. For some fields within the Illinois state government, the degree requirement can be waived for experience.

Re:If I could do it all over again...

[allacds]

I think that anyone who wants an engineering degree for the money will be disappointed. I have a degree in chemical engineering, and I make $55,000 (that is with 10 months of experience). That sounds like a lot for being just out of school, but given the extra effort of obtaining the degree, and the amount of work that is expected from me at my job, I don't think it's a better deal than a liberal arts degree would've been. I think that the value of any degree is what you do with it. If you work to gain valuable experience, advocate yourself, and work well with others, you can make a 6 figure income with any degree.

Yes but consider this. The average starting salary of a liberal arts degree holder is generally quoted as $30-35k. The average starting salary of an engineering degree holder is generally quoted in the $50-55k range. That's a pretty significant difference - the engineering degree yields a return of 40 to 80% straight off the bat. Granted the engineering degree is harder but I'd say it's well worth the effort.

Now that said, I do agree with you that a degree is only as valuable as you make it (to a certain point). But I think taking these numbers in the aggregate probably cuts a lot of the variance due to super high- and low- achievers.

wrong.

[Angry Black Man]

That's correct, the scores are normalized so that the distributions are the same. This means you *can't* compare scores across years. If you did, you would find that, amazingly, the distributions were the same. But have the students stayed the same? Nope. Have the questions stayed the same? No again.

You are wrong. Each SAT has a section that doesnt count for that year's grading but is for future tests. So lets say I take the SAT and get a 1400, and then on the experimental section I get 80% of the questions right. Well, this experimental section will be used next year as part of the test, and a correlation will be done where 80% correct = 1400.

So it is not normalized based on the year you take it. It is correlated to the kids who took it last year, and what score they got. Of course, its a much more complicated algorithm spread out over millions of students. But, in essence, the questions *are the same* and the statistics are mind numbing so that it stays fair. Its actually a very smart system.

Two points (cheating and cheating)

[pbooktebo]

The first thing I note is that, for an institution as brilliant as MIT to make an error that increases their ranking seems a bit suspicious. Despite the fact that many readers here see little or no purpose to these rankings, they are horribly influential, and the difference reported is substantial. MIT is good enough to be great without cheating.

The second point is that many schools are very careful when examining foriegn test scores because of cheating supported by the government. It is well-known that many countries actively encourage cheating (which helps the students get grants or acceptance). The school where I was had a watch list and would ignore scores outright from many countries. Makes me wonder whether they still reported these suspect high scores as part of their average (I expect they did).

Re:If I could do it all over again...

[Dr. Smoove]

I think you're trolling, or you live in the woods in Alaska. No way that someone with years of experience couldn't land a job in the computer field. If I can get job offer after job offer by just sitting on my ass after I post my resume on two sites, then you need to work on your resume skills, or your social skills. One of them is srsly out of tune.