The Myth about “Predictive” Measures
Posted Thursday, August 23, 2007 by Anne-Evan WilliamsFiled under: Reading, Assessments, Administrators, Teachers, Experts, Paolo Martin, Reading Specialist, Richard Capone, CEO, Let’s Go Learn,
by Richard Capone and Paolo Martin
Measures like DIBELS, SRI, or MAPS are the result of schools and districts wanting to predict how their students will do on specific state standardized tests. Will scores go up this year? Will they go down? The companies that publish these tests focus much of their energy on studies that show how scores compare to a "normalized" population, as well as on potentially specific assessments.
The downside of this type of data is that it is less diagnostic—that is, it has limited value in guiding the teacher in how to specifically work with individual students. The backlash against DIBELS has resulted primarily from a misuse of the assessment. But the assessment itself, if used in the proper fashion, is legitimate.
Let me quickly illustrate this. When looking at a large sample of students that we define as the "norm," we generally see attributes that dominate. For instance, the norm population may be English natives; they may have parents with a high school or higher level of education; they may have a mean family income of X dollars. But now we try to use these predictive assessments to diagnose students who are at risk. Well, generally these students are from poorer families, often are not native English speakers, and may have parents with less formal education, if any at all. So to make diagnostic conclusions based on a predictive measure using a norm that is dramatically different doesn’t make sense.
The solution is to use multiple measures that are evidence based. What can the child do well? Look at multiple areas in reading. Stop drawing conclusions from inadequate data. How is the child in sight words, word recognition, phonics, phonemic awareness, vocabulary, spelling, reading comprehension, etc.? Get this data, and then make conclusions on the individual student level.
This is what the DORA (Diagnostic Online Reading Assessment) does. It follows the model that reading specialists take when trying to diagnose students themselves. They look at the student individually and gather as much information as possible so they can prescribe a customized learning path for each child.
The myth that needs to be dispelled is that predictive measures can diagnose students as effectively as evidence-based diagnostic assessments. This simply is not the case. Yes, they can be partially diagnostic. But they were designed to look at larger samples of data. Just because data is valid across a large sample doesn’t make it valid for a single student or for a classroom of students.
(1) Comments • (0) Trackbacks • Permalink
Very refreshing. I agree there is a big mis-understanding among administrators about types of assessment. They focus too much on benchmark testing thinking that they will guide instruction and help students.