Drunk driving

We had a staff meeting at my school on Friday. Classroom teachers received forms for sorting their students into Tier 1, 2 and 3 groups for implementing Response to Intervention, or RTI. K-2 grades will use DIBELS data to sort students, and grades 3 - 8 will use Scantron data. Since I work on organizing the Scantron testing at my school, and end up sifting through the data, I was especially interested in the choice of Scantron data to slot students into RTI tiers.

The form passed out came from our Area office, and uses Scantron National Percentile Rankings (NPR) for sorting. Students in the 24th percentile and up fall into Tier 1 (the lowest priority for special interventions). Students between the 11th and 24th percentiles fall into Tier 2, and students under the 11th percentile fall into Tier 3. (I may be off by a percentile or two -- I'm doing this from memory.)

I have written about the problems of standardized testing and "data-based decision making" before, likening it to the drunk looking for his car keys under a streetlight, not because that is where he dropped them, but because that is where the light is. The light in this case is the pale beam of standardized test data -- it doesn't tell us nearly enough about the student, and for many of the neediest students, it is hopelessly distorting. We won't find the keys to student success with the data, but it's easy for administrators to collect, and to pretend that it means much more than it does. Drunk on data, and wandering off course from the outset.

For example: After we completed the math testing this Spring, about 8 percent of students at my school dropped more that 100 points from their Fall scores. The numbers were worse for reading -- almost 11 percent of students had dropped more that 100 points. These are big numbers, well outside of the statistical error range. Excepting epidemic of brain injury, such a drop can only be attributed to subjective student factors: boredom, disinterest, a desire to be done with test, difficulty with the computer medium, etc.

The belief that these initial numbers were faulty was confirmed when we made students retake the test. Most of the re-takers did much better, erasing most of the drop, and in many cases swinging into solid gains for the year.

The point, again, is that testing students is not the same as taking their temperature, and so much depends on the subjective factor. But the RTI strategy doesn't appreciate the subjective factor -- it's all about the data, as goofy as it may be. One student at my school went from dropping over 200 points, landing in the 7th percentile, to increasing over 50 points for the year (not a great gain, but keeping him at grade level) after he retook the test -- moving from Tier 3 (deserving special interventions), to Tier 1 (no special interventions). [Value-added alert!] It should be said that the possibility of testing into a higher percentile is not very likely, so the general danger is that students will be assigned to tiers they should not be in, consuming time and resources that perhaps should go to needier students.

One of the ironies in all of this is that all of teachers I have talked to already have a sense of their students abilities, and could quickly categorize their students without the Scantron numbers in front of them. After all, they assess their students every day. And the teachers can tell which numbers are off. And all would probably say that the time and the resources to provide interventions to all the students that really them is just not nearly enough; and the process to get the help needed is too long and places too much burden on the already over-stretched teacher.

jd