I took the SAT so you don't have to. It's a very poor test of math skills.

As I write this, tens of thousands of high school students are hunched over desks, filling in little circles with number 2 pencils, laboring to complete the SAT, a test that will have an outsized impact on where they go to college. In my state, the test starts at the teenager-unfriendly hour of 8:00am and last a grueling four hours or more.

One of my daughters is among those students. and as preparation she took the four practice tests provided by the College Board–the private company that owns the tests. (The tests are administered by the Educational Testing Service.) There are two main parts to the SAT, a math test and a verbal test, and as every student knows, the scores range from 200 to 800 on each part. For many decades, this test has been one of the main gatekeepers to college: the US News College rankings use it, and colleges advertise the average SATs of their freshman classes. Every student wants to know what SAT score they need to get into their preferred university. 1.7 million high school students will take the SAT this year, and many of them will take it twice.

I wanted to understand what the test was like, so I took the math test with my daughter–three times. On three successive weekends, we each took one of the practice tests, and then used the answers provided by the College Board to score ourselves and review what we got wrong.

Here's what I learned from taking the SAT math test: it's all about speed. The concepts are not difficult; you need to know algebra, geometry, a little bit of trigonometry, and a tiny bit of statistics. The main skill you need, though, is speed. It's a very poor test of how well you understand math. For the three tests, I was only able to finish everything on time once. Even so, I had to work very quickly and I didn't have time to go back and check my answers.

Question 27 from SAT test 2 from the College
Board. "D" is the correct answer.
The math test has two parts: one with 20 questions, for which you get 25 minutes, and another with 38 questions, for which you get 55 minutes and where you're allowed to use a calculator. The test is designed to trip you up if you work too quickly: many of the multiple-choice answers match the answer you would get if you made a careless error of a particular type.

Doing well on the SAT requires that you know the tricks of the test, and that you've memorized many formulas so that they come to mind instantly. And I mean instantly: if you have to think for 30 seconds to remember something, that's far too long.

What's more, the questions themselves can be lengthy, and students might waste precious minutes just trying to be sure they understand the wording. For example, one question shown here filled half a page: just reading it would take some students longer than they can afford for this speed-obsessed test. (If you want to see a full-sized image, get the tests here.)

Statistics is a relatively new topic area for the SAT, and if the practice test is any guide, they haven't yet figured out how to construct good stat questions.  Here is one of them:

A researcher conducted a survey to determine whether people in a certain large town prefer watching sports on television to attending the sporting event. The researcher asked 117 people who visited a local restaurant on a Saturday, and 7 people refused to respond. Which of the following factors makes it least likely that a reliable conclusion can be drawn about the sports-watching preferences of all people in the town?
A)  Sample size
B)  Population size
C)  The number of people who refused to respond
D)  Where the survey was given 
The official correct answer is D, because (says the College Board) the survey was not collected from a random sample. However, I could argue that A is at least as good an answer, because 117 people is a tiny sample from what is called a "large town," and because we don't know that the people who visit this restaurant are un-representative of the town. Whether you think the answer is A or D, this is a lousy question to put on a test where the answers should be unambiguous.

You might be wondering what my score was. I'm not going to reveal that, but I will say that I had a higher score when I took the SAT in 1975, as a 15-year-old high school student. I must have been faster then, but I'm pretty certain that I understand math better now, after 35 years of working in a mathematical field. I suspect that the current test puts a greater emphasis on speed than the 1975 version, but there's no way to check that without copies of the 1970s-era SAT exams.

Let me put this another way: there's not a single question on any of the practice math SAT exams that I would call difficult. Most of them are quite easy if you know a bit of algebra and geometry. But unless you are fast, answering all 58 questions in 80 minutes is darn near impossible. For example, consider this simple problem:

At a lunch stand, each hamburger has 50 more calories than each order of fries. If 2 hamburgers and 3 orders of fries have a total of 1700 calories, how many calories does a hamburger have? 
This question is not multiple choice; you have to write down a number, which means you have to work it through. Any student who knows basic algebra should be able to solve this, but can s/he do it in less than 75 seconds? And if takes 90 seconds, does that mean s/he should be rejected by Yale?

As a measure of true understanding, the SAT math test is terrible. We should not be using it to make enormously consequential decisions about where almost every high school senior in the country goes to college. On a positive note, a growing number of colleges have rebelled and no longer require the SAT. Two years ago, a large study showed that, at colleges in this group, there was no difference in college performance between students who submitted SAT scores and those who didn't.

Perhaps pressure from colleges that are making the SAT optional will force the College Board to create a math test that measures something that matters for college success. They could go a long way towards a better test by simply giving students twice as much time. If that made the test too long, they could simply ask fewer questions. Meanwhile, I hope that more colleges will make the SAT optional, and instead use high school grades and other, more meaningful measures of a student's knowledge.

(*This was written on June 4, 2016, the date of the second SAT test in the "new" format. Prior to March 2016, the test had three parts, each with a maximum score of 800.)

No comments:

Post a Comment

Markup Key:
- <b>bold</b> = bold
- <i>italic</i> = italic
- <a href="http://www.fieldofscience.com/">FoS</a> = FoS