In my last post I looked at student growth percentiles, a new kind of data that the Washington State Office of Superintendent of Public Instruction. This post follows up by going into more detail about how SGPs are calculated. Back to the simple video with cartoon Anthony!
Cartoon Anthony got an MSP reading score of 344 in 3rd grade and a score of 381 in 4th grade. His score grew in that time. To get his SGP, they took the amount of growth and compared it to a group of 3rd grade students who began with that same score. His score of 381 was higher than 80% of the students in his comparison group, so he gets an SGP of 80. That shows “high growth.”
I’m going to set aside for the moment a couple of really large questions. First, exactly what does the MSP reading score measure? Second, what would have happened to the SGP if Cartoon Anthony was instead compared to a different set of students?
I’m just going to pretend those other questions don’t exist and ask, “How accurate is the SGP?” There are different ways to look at the answer to this question, but I’ll just give the easiest one here. Hang on to your hats, because I’m going to use just an eentsy weentsy bit of algebra. I’m going to use the variable x to indicate the absolute measurement error of a single MSP score. I’d use the real number, but I don’t know it and I couldn’t find it online anywhere.
The change in Anthony’s MSP scores is 381-344, or 37.
The absolute measurement error for each test is x.
Using the simple method (that doesn’t rely on calculating standard deviations), the absolute measurement error for the change in scores is 2x.
So let’s suppose x is 10. That would make the absolute error 20, which means the change in scores could be anything from 17 to 47. Pretty big difference there!
If the measurement error is high enough, it makes Anthony’s SGP meaningless.
Is it meaningless? I don’t know. But the question needs asking.
Kristin Ann King 