There are three reasons. First, before grade 7, many students' ability to express themselves in writing lags behind their ability to express themselves in other ways. (Even in high school, some students—especially those in schools where teachers don't have the time to assign and mark written assignments—lack the writing skills required to convey their ideas clearly.)
Second, DiscoTest rubrics are designed for students between the ages of 12 and 18. They will not do a good job capturing the reasoning skills of many students in lower or higher grades.
Third, although we study how conceptions develop from age 5 through adulthood, we have not yet figured out how to create coding rubrics that work well beyond the early phases of level 11. The rubrics only go up to phase 11:1, the highest phase we expect to see in high school (with a few exceptions). In college, the risk that a student will hit this assessment ceiling increases, so we don't recommend using DiscoTests in this group.
A few educators have suggested to us that DiscoTests are not fair because some students can write better than others. We prefer to think of DiscoTests as important opportunities for students to hone their writing skills—not necessarily their punctuation and grammar, but their ability to make a coherent argument. We regard this as an essential life skill.
When teachers register students to take DiscoTests, we ask them to rate each students' ability to express him or herself in writing. Then, in each report, we remind the teacher that every student performance is influenced by multiple factors, including limitations in his or her writing skills. Like any other assessments, DiscoTests should be regarded as one of many tools teachers use to evaluate student learning.
Students with disabilities: There is nothing wrong with encouraging students who have serious writing difficulties to dictate their responses to DiscoTest questions.
Nothing, if they are used appropriately. They are good for assessing students' content knowledge and skills for remembering and applying formulas. Unfortunately, they are not good for assessing understanding or reasoning, yet most classroom assessment and virtually all standardized assessments use the multiple choice format. Research shows that as many as half of the students who get a multiple choice test item right cannot demonstrate understanding of the problem posed in the item (Schoenfeld, 2007). In this time of No Child Left Behind and high stakes, multiple choice testing, this is a serious issue. To the extent that we "teach to" these tests we under-prepare our students for the challenges of 21st century life by focusing too much on content and not enough on reasoning and understanding.
When students are introduced to a new idea, they must find a way to fit new information with their existing knowledge. Sometimes we ask them to add new ideas that fit well with what they already know, as when we provide them with a new example of a phenomenon with which they are familiar. At other times, we ask them to literally "change their minds." This is the case with many concepts in science, especially when a word students already use takes on a new meaning.
Learning in this situation is HARD, because it asks students to change the way they think. It can take a great deal of evidence to rewire the brain to accept a new way of thinking, and if the new way of thinking is more complex than the student's current way of thinking, it can take considerable time.
When there is not enough time to repeatedly engage the evidence, students often simply memorize a new definition and learn when it is appropriate to use it. When this happens, there is a disconnect between what the student knows (a definition) and what she understands (a concept). So, when asked to explain how she thinks about the phenomenon described by a term, the student falls back on her pre-existing way of thinking. This is where misconceptions come from and why they are so stubborn. (By the way, we like the term preconceptions better than the term misconceptions)
DiscoTests help students learn by applying new ideas to real world scenarios. They also bring preconceptions to light, helping teachers see when students need more evidence and experience to change their minds.
Every menu in a DiscoTest coding interface includes an option like, "did not mention ____." This is what we mean by the "missing" catagory.
Right now, we don't use the "missing" category whe we calculate a score. As long as students do their best on assessments and have no learning disabilities or limitations that affect their ability to write responses, accurate scores can be calculated without it.
However, we think missing counts could provide teachers with valuable information about student performance. For example, we think that a high number of " missing" codes in phases 10:3 and 10:4 (elaborated and highly elaborated) may indicate that a student is not doing a good job showing off everything he or she understands. It is also possible that "missing" counts will be able to help us tell teachers which concepts students most need to work on.
Because we don't have a lot of empirical evidence about the meaning of missing responses, right now we are using the missing category to collect data. Keep your eyes open, though, they may turn up in DiscoTest reports in the not-too-distant future.
