CPS moves to online assessments

I attended a briefing for principals and test coordinators on the CPS district-wide assessment plan for the coming year. The briefing was given folks from the Department of Student Assessment, a part of the Office of Research, Evaluation and Accountability. Click here for a link to the Powerpoint presentation for elementary schools. The DoSA page also has a link to the high school strategy presentation.

I was primarily interested in Grades 3 - 8, so this summary only touches on that. CPS will administer three benchmarks assessments in reading and math this year, in October, January and May. (Here's a link for the complete assessment schedule.) The district is moving towards an online benchmark assessment, and is supposed to be working to get all schools ready to move to all-online benchmark assessments in 2010-11. Some schools (if they have the technology infrastructure) will be able to start with online assessment beginning with the winter assessment in January. (Constructed response questions will still be graded by hand, by teachers, and entered manually into CIM.)

Setting aside the tremendous limitations of multiple choice as an assessment tool (and how education, like so many things today, is inverted to serve the available technology, and not the other way round, but that's another discussion) ...

The advantage to online assessment is that multiple choice results are available immediately, instead of two to three weeks later. The disadvantages: getting the students used to the testing format and software (sigh, that this should be an issue -- we train them to be multiple-choice test-takers), scheduling test time in the computer lab, the inevitable network and software and computer problems, the extra burden on technology staff, handling accommodations in a computer lab setting, and no doubt other issues not immediately popping into my brain.

CPS is also moving to a second testing format, the Scantron Performance Series, which is a computer-based adaptive testing tool. "Adaptive" means that the testing software presents the test-taker with more difficult questions if he or she answered the previous question correctly; or an easier question if the previous question was answered incorrectly, until the testing algorithm can determine a performance level for the test taker. Students will take reading, math and science Scantron tests.

In 2009-10, Scantron is available as either an addition to the district benchmark assessment, or as a substitute. In 2010-11, students will take both tests, three times a year. This flows from the new Chicago Public Schools CEO Ron Huberman's emphasis on data collection as an education management tool. See Catalyst's 9/3/09 posting "Huberman outlines strategies to improve Chicago schools" for a brief overview.

The Benchmark Assessment is a criterion-referenced assessment, and so is intended to measure student mastery of particular skills (the criteria). [The math assessment question mix is also mapped to the Chicago Math and Science Initiative (CMSI) pacing guide for the four chosen CMSI math curricula (Trailblazers, Everyday Math, Connected Math and MathThematics).] As a performance measurement tool, the Benchmark Assessment is not so useful. The best proxy in the BA data for tracking overall performance has been the ISAT Predicted Scale Score, a guesstimate of how the student will perform on the state standardized test. The predicted score is supposedly anywhere from about 75% to 90% accurate (I just got our schools numbers so haven't seen yet how this relates to our results.)

The Scantron test is a performance measurement -- it doesn't say specifically what a student knows or needs to work on (which isn't to say necessarily that the Benchmark Assessment does, although that is its purpose), only where a student is in the mass of students, and is he or she making "progress" based on prior Scantron scores. The Benchmark Assessment is considered "low stakes" (no one's job is on the line, schools won't be closed because of scores, students won't be held back -- as opposed to "high stakes" like ISAT where all three of the above are the case).

I suspect that the Scantron test will come to be considered by school administrators and teachers as high stakes. I am guessing that the Scantron numbers will be the bottom line numbers to determine if students are "progressing", and therefore used by top-level administration to lean on principals and teachers.

jd

P.S. Writing assessments are built into the Reading Benchmark (3x a year, scored by teachers) plus one additional district-wide assessment, scored externally, that will be used for promotion determination. K-2 students will use one of three early literacy assessment tools (including DIBELS).