Why Do Test Scores Go Up and Down? (Consistency is Hard)

I often deemphasize the score from individual practice tests, and I worry that this makes parents suspicious. “Like jazz, it’s about all the mistakes they didn’t make!” 

Here are some actual reasons your child’s score could vary from test-to-test — all of which address the challenges to being consistent:

First, students will leave practice one day as conquering champions, and then at home, days later, they will completely blank on how they were just doing these questions. It happens. For a lot of students, the strategies start as weird suggestions. Even once the strategies make sense, students will abandon them as soon as they feel anxious or as soon as they feel confident. The work they do at home is not testing them to see how many they can get right or wrong; it’s “just” practice. “Do this same series of steps until it becomes habit” is how we build consistency. Until these steps become habits, students’ performance will vary based on how they’re feeling that day and how they choose to approach individual questions.

Additionally, as much as the standardized test writers work to make the tests consistent, they’re not perfect at it. The Verbal section is probably the least consistent part of the whole test. So much depends on which words they decide to include on a given test. For students, it can be a weak consolation to point out that if it’s harder for them, it’s harder for everyone else, too. But even on the Reading and Math sections, the tests vary dramatically. One SSAT reading section starts with some obtuse 16th-century philosophy, and continues with tough texts and bad questions. (I have a personal vendetta against this SSAT, and I apologize to anyone who has asked me a simple question and received a long rant about that particular test.) 

Even worse, both the SSAT and ISEE (and possibly the HSPT) have dramatically changed their Math sections. If you look at the material they have online, and talking to students who have taken the latest tests, the Math section now includes Algebra and more advanced Math concepts. This means the older practice tests that SSAT and ISEE have released aren’t accurate reflections of what the Math sections are now. The most accurate versions of the current Math sections we have are from a few online versions. When it comes to commercial test guides, finding consistency is even harder: one company’s practice tests will have more accurate math sections, while another company will have more accurate reading sections. All the practice tests are useful, but they’re imitations, and sometimes contain errors. The scores won’t be consistent because the tests often aren’t.

Moreover, the SSAT, ISEE, and HSPT each have their own way of calculating the percentiles, all with their own issues and advantages. When it comes to scoring the practice tests, we’re a bit more limited. Of the three, the SSAT provides the most useful information about scoring, scaling, and converting to percentiles. Overall, the test-makers keeping much of their data private is yet another reason that students’ practice scores vary from test to test.