http://www.letsgolearn.com/faqs/ en aekwilliams@LetsGoLearn.com Copyright 2009 2009-11-18T18:39:20+00:00 http://www.letsgolearn.com/faqs/why_doesnt_lgl_provide_formulas_for_geometry_questions_in_doma_pre-algebra/ http://www.letsgolearn.com/site/why_doesnt_lgl_provide_formulas_for_geometry_questions_in_doma_pre-algebra/#When:18:39:20Z We have chosen NOT to include formulas because some parts of the country require students to have memorized them for high stakes assessment, and some provide them.  If we provide the formulas, than the parts of the country that still require memorization do not get an accurate assessment of student knowledge.  Teachers may choose, however, to provide students with formulas if their district does not require students to memorize them.  Math Assessment FAQs 2009-11-18T18:39:20+00:00 http://www.letsgolearn.com/faqs/why_are_the_fluency_passages_fiction/ http://www.letsgolearn.com/site/why_are_the_fluency_passages_fiction/#When:18:29:08Z When we designed the fluency passages, we chose fiction because generally we expect fiction to be more conversational than non-fiction, with more dialogue, and would lend itself more diagnostically to analyzing prosody.  Furthermore, with fictional fluency passages and non-fiction used in the comprehension subtest, DORA addresses the unique reading differences between fiction and non-fiction, making it a more robust assessment.  Assessment FAQs 2009-11-18T18:29:08+00:00 http://www.letsgolearn.com/faqs/why_do_individual_dora_measures_vary_in_range_for_individual_students/ http://www.letsgolearn.com/site/why_do_individual_dora_measures_vary_in_range_for_individual_students/#When:23:20:49Z This FAQ answers multiple questions:Why are some of Let’s Go Learn’s DORA scores unexpected?Why do some scores seem very high or very low?When using a computer-mediated diagnostic, formative assessment, like DORA, it is not uncommon to see scores that might look a little different than behavior that you might see in the classroom.  In this document, Let’s Go Learn attempts to address some of the most common misunderstandings about DORA scoring.High Scores on the Word Recognition SubtestThe Word Recognition subtest assesses students’ abilities in decoding skills, using a combination of criterion-referenced real words and phonetically regular invented words.  Words are presented to students orally, and they are asked to identify the correct word from four choices.  While this subtest does accurately assess students’ word recognition ability, it is an out-of-context activity, with an oral component.  As such, teachers may sometimes see unexpectedly high scores from students who have strong decoding abilities, but perhaps low comprehension scores.  While students can use decoding skills to isolate the correct word, they may still struggle to read and comprehend the word in context.  This test should be used to evaluate decoding and word analysis skills in isolation, but is not an accurate assessment of contextual reading skills.These scores, however, are usually an accurate reflection of students’ graphophonic skills.  The reliability of this subtest is particularly high (grade level delta = 0.19, SE = 0.12), meaning that when the test is administered repeatedly with no instructional time between assessments, students will score, on average, a difference of less than two school months between assessments.  Also, this subtest was correlated to both the Diagnostic Assessment of Reading (Riverside) (r = 0.81) and the Woodcock Word Identification Test (r = 0.92), both with statistically and practically significant levels of correlation, indicating very high levels of validity for the subtest.High Scores on the Word Meaning SubtestIt is important to note, when examining the scores on the Word Meaning subtest, that this subtest does not assess students’ reading vocabulary (or words that they can both read and define).  Instead, it assesses students’ oral vocabulary, as oral vocabulary is often considered predicative of students’ future reading ability.  Students cannot read words that do not exist in their oral vocabularies, so an assessment of oral vocabulary would help identify a gap that would prohibit students from reading achievement.  It is a particularly important subtest for second language learners and for students with developmental language delays.  For other students, however, the word meaning subtest may appear to give unexpectedly high scores.These scores, however, are routinely not errors.  The Word Meaning subtest has been shown to be reliable, with a statistically significant test-retest correlation (r = .60, SE = 0.19).  Further, it has been correlated to the word meaning subtest of the Diagnostic Assessment of Reading (DAR), with a moderate to high level of correlation (r - .60).  Further, in 2003, items in the Word Meaning subtest underwent major revision to ensure that test scores were not consistently higher than expected.Low Spelling ScoresSpelling is the most challenging sub-test on DORA as the answers are completely student generated as opposed to multiple-choice. If students are performing well on classroom spelling tests, consider the difference in the task. If on Monday they are given a list of 10-25 words (depending on the teacher), they spend the entire week memorizing those words:  writing, re-writing, creating flash cards, drilling themselves, drilling their peers, having sample tests in class and at home over breakfast. The students are given many opportunities to memorize those words and most will do well on the Friday spelling test. When they come take DORA, they are seeing words that they have not just spent a week practicing, and they have only one chance to spell the word correctly. It is a sample of how the student is spelling, in general, without any practice.  Do you administer a spelling pre-test at the beginning of the week? Do your students get the same score on Monday as they do on Friday? Probably not, but I would look at the Monday score as an indicator of how well the child really spells. With practice children will learn to memorize and spell words.  Continued work on classroom spelling lists will improve students’ diagnostic spelling scores, as they continue to be exposed to more words and more complex spelling patterns.Let’s Go Learn’s Spelling subtest has been correlated to two other nationally recognized spelling assessments: the spelling subtest of the Diagnostic Assessment of Reading (DAR) and the spelling subtest of the Wide Range Achievement Test (WRAT). Both studies indicated a statistically significant correlation to both assessments, assuring the validity of the DORA spelling subtest.Low Comprehension ScoresMany factors affect a student’s ability to successfully comprehend a text.  Some students struggle with decoding the text they encounter or with the language structures (i.e., phrases and idioms) used.  Other students may possess limited background knowledge about the topic of the text or they may not be interested in what they’re reading.  While Let’s Go Learn’s comprehension test presents students with non-fiction topics that they are likely to have encountered in school, some groups of students may have less familiarity with the subject matter in DORA than in other comprehension assessments. Using non-fiction passages with topics taught in most classrooms across the nation provides less variability in assessment results.  The language involved in generating non-fiction passages is easier to standardize, as it does not contain conversational colloquialisms that are often regionalized in the U.S.  Also, non-fiction passages offer a range of topics common to many classrooms, reducing bias due to race, gender, and culture.  While non-fiction is sometimes more difficult for children to read than fiction, Let’s Go Learn has made a conscious effort to control for this by writing comprehension questions that are not too difficult and by creating an administration protocol which ensures that children only see questions within their comfort level as the sub-test raises and lowers the difficulty of passages according to success on DORA. Another factor that can make scores on DORA seem lower is if your students have been tested using traditional teacher-mediated, pen-and-paper assessments.  On these assessments there is more room for discrepancy, as teachers often ask follow-up questions to clarify students’ responses and students often become familiar with the administration protocol.  Let’s Go Learn’s DORA removes some of this variability often associated with teacher-mediated assessments.  Often comprehension tests, like those utilized in annual state assessments, allow students to re-read the passage after they have seen the questions.  This type of assessment can lead to false positive scores, as students learn strategies for skimming that may not be an indication of absolute comprehension ability. Allowing students to re-read passages introduces a new variable to the assessment that is difficult to control for.  That is, some students choose to re-read the passage over again while other students choose not to re-read the passage.  Allowing students to re-read a passage thus increases the variability of the comprehension sub-test score.  By allowing students to read the passages only once, DORA provides a better indicator of how well students will perform in realistic reading situations.  This gets back to the purpose of DORA, which is to provide diagnostic data for teachers to guide instruction, but could consequently result in scores on the comprehension subtest that are lower than teachers or parents might expect.Also, because DORA is criterion-referenced-that is, based on a set of criteria identified by experts-it is possible that the items might differ from other criterion-referenced assessments you may have encountered.  This does not preclude the utility or meaningful information produced by DORA‘s comprehension sub-test.  It just means that one must consider its difficulty relative to other available comprehension tests.  The avoidance of false positives, as mentioned in the previous question, is also a factor that can make scores appear lower. If other comprehension measures used in the past have a lower degree of false positive aversion, then the difference when comparing DORA to this other measure may appear significant.  Our philosophy is that it is worth it to avoid incorrectly labeling a low comprehension student as high, even if it means on occasion labeling a high comprehension student as slightly lower than his or her real ability.  And have no doubt, comprehension measures must choose one or the other possibility.  There is no way to avoid biases.  Another factor that should be considered is the student’s motivation.  Longer assessments do run a higher risk of fatiguing the student.  And the factor that causes the greatest test score variance is student motivation.  Therefore, students need to be properly introduced to the idea of DORA.  Teachers should stress that this assessment will help them do a better job of instructing the students.  Also, the assessment should be broken up into manageable sessions and students should be monitored during testing.  If some students seem fatigued, the teacher should consider stopping the assessment and resuming it later.The Comprehension subtest scores have also been validated, to ensure that the scores that a student receives are not abnormally high or low.  In test-retest analysis, when students took DORA comprehension test repeatedly, the grade level delta scores was 0.35 (SD = 0.13).  In other words, when students retake the test, 95% of students’ scores will have a difference of between 0.09 and 0.61 grade levels; almost all students will score less than half a grade level differently.  This indicates the consistency of DORA’s comprehension subtest.  Further, the comprehension subtest has been correlated to both the Diagnostic Assessment of Reading (DAR) comprehension subtest, and the Gray Oral Reading Test, with both indicating medium-high to high levels of correlation to these other assessments.In summary, many factors might make it appear, on occasion, that students’ scores on DORA‘s Silent Reading sub-test are incorrectly lower than their reading ability compared to other reading measures.  However, when examining the biases of each measure, the statistical soundness of DORA’s subtest validity, and interpreting DORA for what it seeks to do, these discrepancies, if any, can usually be explained or accounted for.  Furthermore, there is low probability that any discrepancy between measures will be large enough to negatively affect any particular student’s instructional plan. Reports FAQs 2009-10-23T23:20:49+00:00 http://www.letsgolearn.com/faqs/why_are_doma_basic_math_skills_reports_different_from_dora_reports/ http://www.letsgolearn.com/site/why_are_doma_basic_math_skills_reports_different_from_dora_reports/#When:03:03:49Z DOMA Basic Math Skills is a diagnostic assessment, much like DORA. However, because the cognitive processes of quantitative reasoning are so different from the cognitive processes of reading, it cannot be entirely like DORA, and the reports generated are consequently different.  In math, it happens that the cognitive processes tend to be more linear in nature, particularly when approached from the perspective of the strands outlined in Principles and Standards of School Mathematics, developed by NCTM .  DORA is based on the three cueing system model of reading in which there are three discreet cognitive areas that must function together in order to facilitate strategic reading. When we test those three areas, we get a very clear sense of each student’s strengths and weaknesses.  DOMA, however, is based on the model of mathematics proposed by NCTM, dividing quantitative reasoning into strands that also must function together for complete quantitative reasoning.  According to NCTM, “The mathematical Content and Process Standards…are inextricably linked.”  In a way, we can think of these six strands (Numbers and Operations, Algebra, Geometry, Measurement, Data Analysis and Probability, and Problem Solving) as a type of cueing system for quantitative reasoning.  By assessing students in math linearly within each strand, we are able to identify student strengths and weaknesses that paint a complete picture of mathematics ability.  When there is a hole in one strand, there is a gap in student achievement.  So while both assessments are diagnostic, they are based on different models of the cognitive processes they assess; the reporting must reflect those models and, therefore, cannot be the same.In DOMA Basic Math Skills, only two of the NCTM strands are assessed.  According to NCTM, “School mathematics programs should not address every topic every year.  Instead, students will reach certain levels of conceptual understanding and procedural fluency when they plan their mathematics instruction.”  Let’s Go Learn has chosen to focus DOMA Basic Math Skills on two strands on which elementary math education often focuses instruction.In DOMA Basic Math Skills, the detailed list of all the constructs tested indicates where you need to work with the student.  Let’s Go Learn doesn’t provide additional recommendations because the unchecked boxes in the student’s zone of instruction are exactly what the student needs to work on.Additionally, Let’s Go Learn offers a tool to examine student performance in one strand against performance in others.  The Class Profile Report provides summary paragraphs, identifying traits of students with differing performances across the strands and offering instructional recommendations, much like in the DORA report.The group strand report is also an amazing tool to help differentiate instruction in small groups within a classroom. Here is the video:http://www.letsgolearn.com/virtual_support_tours/view/doma_reporting/1.  National Council of Teachers of Mathematics.  Principles and Standards for School Mathematics. 2000. Assessment FAQs, Reports FAQs 2009-05-25T03:03:49+00:00 http://www.letsgolearn.com/faqs/why_are_reports_being_generated_for_a_different_student/ http://www.letsgolearn.com/site/why_are_reports_being_generated_for_a_different_student/#When:17:02:28Z I was trying to run reports under one students name and I kept generating reports for a different student from a different class. This happened several times. I would like to be assured of the confidential integrity of the information these programs generate about individual students.It sounds like you are experiencing an error in your Acrobat application. The reports that are showing up are probably reports that were previously viewed on the computer that have been cached. In the past when users updated their Acrobat application, this problem went away.We also currently put a random number at the end of each PDF file name that is generated in our system via the web browser. This has eliminated all browser errors so far that ignore the "do not cache" code embedded in our web pages.  Internet Explorer historically has ignored "do not cache" commands that is a part of standard web page coding. Reports FAQs 2009-05-12T17:02:28+00:00 http://www.letsgolearn.com/faqs/do_lets_go_learns_scores_in_reading_comprehension_represent_an_independent_/ http://www.letsgolearn.com/site/do_lets_go_learns_scores_in_reading_comprehension_represent_an_independent_/#When:21:45:14Z Let’s Go Learn’s Diagnostic Online Reading Assessment (DORA) was adapted from reading assessment protocols created by the Cal Reads program, developed by Dr. Richard McCallum at UC Berkeley. These protocols are consistent with traditional Informal Reading Inventories (e.g., Qualitative Reading Inventory, Diagnostic Assessment of Reading, Basic Reading Inventory) utilized by many teachers and reading specialists. The instructional level is defined as the child’s highest level of mastery on the subtest – or the level right below that where he/she reached frustration. To obtain this level, DORA first presents text to students at a level which should be very easy for them to master. If a student masters that text, the next level will be administered. Text of increasing difficulty will continue to be presented until the student fails to achieve mastery. At this point, the student reaches what is called the ”frustration” level. The last text level before the student reached frustration is called the ”instructional” level; and the text level mastered below the instructional level is called the ”independent”  level. The silent reading comprehension score reported on Let’s Go Learn’s DORA is the instructional level.  At this level, the text is not too easy, nor is it too hard; instead, it is the level at which optimal learning should occur with the appropriate instructional support. Some often liken the instructional level to a student’s Zone of Proximal Development coined by psychologist Lev Vygotsky (1978).  Once the instructional reading comprehension levels (which are based on grade equivalencies) are determined by DORA, they are converted to Guided Reading, Reading Recovery, DRA, and Lexile levels for the convenience of teachers using these instructional programs. While these converted levels will have a minimal margin of error (as leveling systems are often not exactly one-to-one correlations), they still represent a student’s instructional level for those leveling systems as determined by DORA. Reading Terminology, Assessment FAQs, Reading Comprehension Test FAQs, Reports FAQs 2009-04-15T21:45:14+00:00 http://www.letsgolearn.com/faqs/why_might_doras_lexile_level_be_different_for_a_student_than_other_assessme/ http://www.letsgolearn.com/site/why_might_doras_lexile_level_be_different_for_a_student_than_other_assessme/#When:21:38:13Z Our silent reading passages were sent to MetaMetrics, and they provided the range of Lexile scores that DORA returns. What is important to remember when comparing a Lexile returned by DORA and a Lexile returned by another assessment, such as SRI, is the type of comprehension test.  Comprehension tests are not all the same, and each one is based on a different definition of comprehension.  At Let’s Go Learn, we have created a comprehension test that uses longer passages, non-fiction, which is often more difficult to comprehend, and does not include embedded vocabulary, meaning we have excluded that additional confound.  SRI uses short passages and embedded vocabulary, so the Lexile returned by this assessment could very well be different.  It’s the type of comprehension passage that we’ve used that makes our Lexile levels different. Assessment FAQs, Reading Comprehension Test FAQs, Reports FAQs 2009-04-15T21:38:13+00:00 http://www.letsgolearn.com/faqs/how_often_can_i_test_my_children/ http://www.letsgolearn.com/site/how_often_can_i_test_my_children/#When:15:28:41Z We recommend that one not assess sooner than 8 to 12 weeks with DORA...I home school my 6 children and I would like to know if I can test them monthly to make sure the curriclum I am using is working. Thank you for your help.We recommend that one not assess sooner than 8 to 12 weeks with DORA.  The reason being that in some areas it is hard to show growth in that short period of time.  For instance vocabulary and reading comprehension.  In these areas there is a certain margin of error because of the nature of these subtests. So if you test too soon, you either may not show growth or sometimes it could go down when because of testing variability. For instances children’s performance with reading comprehenion will vary by topic.  If they like the topic a lot and have good background knowledge about it they may do better.In practice you probably don’t need to administer our assessment as often.  What I would do is run the assessment, look at each child’s profile, and then you can probably come up with an informal way of measuring growth.  i.e. download a sight word list if a particular child is still learning sight words, then keep a record of how long it takes him or her to get through it or how far he or she can get through it.  We also have a paper and pencil fluency test that you can download on the "home" tab when you log in. You can use that to check fluency which is how well a child reads aloud.  That is a timed test too so you end up with a number of words read per minute.Finally another option is for you to add Unique Reader to your account.  It is very affordable for a family.  It gives you great supplemental differentiated instruction.  And the last activity of each lesson is a graded review activity so you’ll get a score that is logged.  This is good for progress monitoring as well.http://www.letsgolearn.com/lglsite/public_school_products/Unique_Reader/You can check out Unique Reader with this link.Good luck and thank you for your interest in DORA. Assessment FAQs 2008-12-12T15:28:41+00:00 http://www.letsgolearn.com/faqs/what_is_the_difference_between_a_formative_assessment_and_a_summative_asses/ http://www.letsgolearn.com/site/what_is_the_difference_between_a_formative_assessment_and_a_summative_asses/#When:21:29:00Z Formative assessment is an ongoing assessment used to inform instruction...What is the Difference between a Formative Assessment and a Summative Assessment?Formative assessment is an ongoing assessment used to inform instruction. The frequency is not defined, so it might be weekly or quarterly. Also what type of measure is not defined as long as the information is helpful for the teacher in making adjustments to instruction in an ongoing manner. It could be as simple as classroom observations, a fluency test, or full diagnostic assessment like DORA.Summative assessment is given a pre-determined intervals. It could be anything from a state assessment, benchmark test, or end of unit test. It specifically is testing the knowledge that a student should have mastered at a particular point in time. It is used for accountability. Reading Terminology, Assessment FAQs 2008-07-01T21:29:00+00:00 http://www.letsgolearn.com/faqs/what_makes_an_assessment_diagnostic/ http://www.letsgolearn.com/site/what_makes_an_assessment_diagnostic/#When:21:22:01Z These are deep assessments that ask additional questions to find out how the teacher should improve instruction for a student or students...What Makes an Assessment Diagnostic?These are deep assessments that ask additional questions to find out how the teacher should improve instruction for a student or students. Diagnostic assessments look beyond student achievement, to examine patterns of strengths and weaknesses and diagnose the WHY of student achievement and not just the WHAT.  Generally speaking short assessments are rarely very diagnostic.  Reading Terminology, Assessment FAQs 2008-07-01T21:22:01+00:00