SAT Won't Report Low Scores

National Review's blog reports that the SAT is changing so that only a student's MAXIMUM score out of all the times he takes the test will be reported to colleges. What amazing favoritism to rich, stupid, applicants!

Or maybe not so amazing. This will be a bonanza for the SAT company, since their tests will be taken so many more times. This is especially true nowadays, when many colleges have merit-based scholarships and your $45 retest fee might have a 1/10 chance of yielding you $1000 extra in tuition breaks.

It also raises an interesting mathematical question. Suppose everyone ends up taking the test exactly 8 times. This will cost a lot more, of course, but will it yield more accurate evaluation of the applicants? Which provides more useful information:

1. A single test score.

2. The maximum of 8 test scores.

The answer depends on the distribution of an individual's test scores for his given talent. If someone with ability X scores X on the test with probability .9 and X-y with probability .1, the Maximum is a better measure (in fact, then it is even better than the average of 8 test scores).

If someone with ability X scores X on the test with probability .8, X-y with probability .1, and X+y with probability .1, which is better? The maximum still, I think. In almost every case, person i will end up with a maximum of Xi+y, and we can simply subtract y and get a person's ability.

If someone with ability X scores X on the test with probability .999 and X+y with probability .001, then I think , the Single reported score is better. It is right with probability .999, whereas the Maximum will frequently be X+y (with probability 1-.999^8) so it will be right with only probability .992. (I haven't phrased that carefully-- what we care about is not the percentage of "right" answers but the variance of the measure minus the true ability, but in this special case the two criteria give the same answer.)

What if the distribution of test scores around ability has a normal distribution? I don't know. The answer might depend on the variance. I'll ask our job candidate at lunch. He's a couple of years out of grad school already, so he shouldn't freak out at the question.