Frequently Asked Questions

Get answers to the most commonly asked questions.
Home > FAQs Home

DORA Comprehension Sub-test and Why Look-Backs Are Not Allowed

Introduction

To many, reading comprehension is simply an understanding of a piece of writing.  However, the notion of what “understanding” really means and the process involved in grasping a piece of text have been debated in education for decades.  Some say that reading comprehension starts with an effortless decoding of the text.  After that, it becomes more complicated to define adequate comprehension and the process a reader goes through to make meaning out of the text.  Experts agree that reading comprehension is a complex socio-cognitive process affected by many domains, including (but not limited to) a person's prior knowledge, motivations for reading a piece of text, activation of schemata, information gathering and processing, and social interactions with different individuals (Lewis, 1993; Rosenblatt, 1994; Ruddell & Unrau, 1994).  Because of the complexity of the factors involved in reading comprehension and the expectations of what readers will take away from a piece of text, there have been different perspectives on how reading comprehension should be assessed.  This short article looks at the issue of whether test-takers should be allowed to look back at passages when answering comprehension questions on reading assessments, how this issue affects the reliability of comprehension assessments, and how Let's Go Learn treats “look-backs” in its Silent Reading Comprehension sub-test.

Look-Backs in Informal Reading Inventories

Informal reading inventories (such as the Qualitative Reading Inventory, The Basic Reading Inventory, The Flynt-Cooter Reading Inventory, and the Diagnostic Assessment of Reading) have become common tools for assessing students' reading abilities.  In most reading inventories, after a student finishes orally or silently reading a passage, the text of the passage is removed before he or she is directed to answer questions.  This procedure is meant to give the assessor an indication of how much information the student retained from the reading of the passage and an indication of how well the student was able to process the main ideas and details of what he or she read.  The main purpose of this way of assessing reading comprehension without looking back at the passage for help is to reliably quantify and qualify how much information a student retained and understood compared to other students under similar test-taking conditions (Kender & Rubenstein, 1977). 

Some educators have argued that prohibiting looking back at the text to help answer comprehension questions penalizes students for short-term memory issues.  They also argue that useful qualitative information is lost when children are not allowed to display their natural sense-making strategies, such as looking back at the text.  Some teacher-mediated IRIs have revised their assessment protocols to give students an option of looking back at the text for help while the teacher notes in what capacity students use look-back strategies (Leslie & Caldwell, 2001; Manzo, Manzo, & Albee, 2004).

Reliability Lost vs. Affordances Gained by Allowing Look-Backs

As argued by some reading experts, allowing students to look back at the text of the reading comprehension assessment gives teachers valuable qualitative information about the test-taking and/or meaning-making strategies of students, and it provides a more authentic picture of how readers engage with text as they examine it for information.  However, allowing look-backs does not necessarily translate into better assessment information about a student's reading abilities.  The issue of an assessment's value often rests on its use and the realities of test administration.  For schools with trained teachers and specialists who can administer one-on-one, teacher-mediated informal inventories, it might make sense to gather more qualitative data about reading behavior, such as students' propensity to ask for assistance, sub-vocalizations, or looking back to the text to help answer comprehension questions.  However, when reporting quantitative data for the purpose of assigning an instructional reading level and for monitoring student progress, allowing students to look back at the text for help in answering comprehension questions introduces another variable which affects the reliability of the quantitative data reported by the assessment.  This is especially true for adaptive, non-teacher-mediated, online assessments like DORA.  Variables like looking back, which cannot be controlled for, introduce a higher level of error in reporting reading levels, as not all students will look back at the passage; furthermore, some students will look back at the passage because they want to clarify how they understood the passage, as opposed to looking back in order to get the right answer (Kletzien, 1991; Kender & Rubenstein, 1992). 

Conclusion

Allowing students to look back at passages introduces a new variable into automated diagnostic reading assessments like DORA.  Rather than reliably assessing the amount of information a student absorbed by reading passages independently, look-backs make it difficult to trust that the numerical reading levels reported by the assessment are valid.  It becomes unclear whether the test assessed how much information a student took away from the passage or, rather, how well he or she hunted for answers in the text.  Allowing students to re-read passages introduces a new variable to the assessment that is difficult to control for:  that is, some students choose to re-read the passage while others choose not to (Leslie & Caldwell, 2001).  Allowing students to re-read a passage introduces a test-administration variable that is difficult to control for on the comprehension sub-test, which produces poor reliability of the information gathered by automated reading assessments like DORA.  For the purposes of providing reliable and diagnostic information on how well a student understood a passage, allowing students to read the passages only once on DORA produces better assessment information.  

References

Kender, J., & Rubenstein, H. (1977).  Recall versus reinspection in IRI comprehension tests. Reading Teacher, 30, 7, 776-79.

Kletzien, S.B. (1991). Strategy use by good and poor comprehenders reading expository text of differing levels. Reading Research Quarterly, 26, 67-86.

 Lewis, C. (1993).  "Give people a chance":  Acknowledging social differences in reading. Language Arts, 70, 454-61.

 Leslie, L., & Caldwell, J. (2001). Qualitative Reading Inventory – 3. New York, NY:  Addison Wesley Longman, Inc.

 Manzo, A.V.,  Manzo, U.C., & Albee, J.J. (2004). Reading assessment for diagnostic-prescriptive teaching.  Canada: Wadsworth.

Rosenblatt, L.M. (1994).  The transactional theory of reading and writing. In R. Ruddell, M. Ruddell, & H. Singer (Eds.), Theoretical models and processes of reading (4th ed.) (pp. 1057-1092).  Newark, DE:  International Reading Association.

 Ruddell, R.B. and Unrau, N.J. (1994). Reading as a meaning-construction process:  The reader, the text, and the teacher. In R. Ruddell, M. Ruddell, & H. Singer (Eds.), Theoretical models and processes of reading (4th ed.) (pp. 996-1056).  Newark, DE:  International Reading Association.

 Zabrucky, K., & Ratner, H.H. (1992). Effects of passage type on comprehension monitoring and recall in good and poor readers. Journal of Reading Behavior, 24, 373-391.

Tags: dora, comprehension, lookback, look-back, look, back, silent reading

Comments
    No comments yet.

Post a new comment:

Name:

Email:

Smileys

Remember my personal information

Notify me of follow-up comments?

Security Challenge:

Type in the following word: teacher

Submit