Working with DiscoTests

DiscoTests are a cross between standardized tests and classroom assessments, in that they are validated like standardized tests, but scored by teachers.

At the core of all DiscoTests are one or more real-world problems or scenarios and a series of 5 or more questions that require short essay responses. Why 5? We have found that 5 is a "golden" number when it comes to getting an accurate picture of how a student is thinking—at least from a psychometric point-of-view. Fewer essays lead to scores that we can't have enough confidence in, too many more essays and students get frustrated. Five is usually just right.

Sometimes DiscoTests include survey or multiple choice questions. We add these when knowing about student preferences or their particular content knowledge will help us evaluate how they are thinking. When such questions are included in teasers, the results are automatically calculated and included in the Student Report.

All DiscoTests are coded by teachers, who select codes from 20-to-30 pull-down menus per test. The coding choices are a lot like descriptions of levels in scoring rubrics. The difference, is that each list of choices in DiscoTest menus focuses on a single theme. Once teachers are familiar with the codes, they can fly through assessments.

For each essay, you can expect 3-8 pull-down menus, depending on the range of themes we anticipate observing in response to a given essay (based on research). For example, questions in the Energy Teaser bring up themes related to force, gravity, kinetic and potential energy, energy forms, transfer or transformation, and conservation. Each pull-down menu is associated with one of these themes.

After teachers have coded the essays, there are places for them to provide the student with personal comments and/or make suggestions for additional coding categories. When they submit their codes, these are analyzed with an algorithm and the results are used to create a student report.

The visitor menu in the left panel of this page includes links to sample assessments, coding interfaces, and reports.

 

DiscoTests are a bit different from conventional tests. Our main reasons for creating them were to (1) provide teachers with information about student development that can help them meet each student's particular learning needs; and (2) make testing a part of learning.

This means that DiscoTests can be used in ways teachers might not use other forms of assessment. For example, DiscoTests can be assigned:

• to groups of students;

• as open-book homework assignments;

• before instruction, to find out what students already understand about a given concept;

• after instruction, to deepen student understanding and evaluate progress; or

• any combination of the above.

We generally offer two forms of an assessment, so teachers can give slightly different assessments before and after a lesson is presented.

Don't be surprised if some students don't seem to do as well on DiscoTests as they do on conventional tests. This is common, particularly if students are unaccustomed to explaining their reasoning in writing.

 

Student reports combine information from several sources. They are made up of two sections, one for students and one for teachers and parents:

The student section includes:

• the student's responses;

• a score, with a brief description of the score's meaning;

• suggestions for specific activities that are likely to help the student build a more complete understanding; and

• the teacher's personal comments.

The teacher/parent section adds:

• information about student thinking at the level of the students' performance; and

• a comparison of the student's score with average expectations at his or her grade level.

The reports also include general suggestions about interpreting developmental assessments.

 

 

DiscoTest scores are based on Fischer's dynamic skill scale, which decribes cognitive development from birth through adulthood, is composed of 5 tiers and 13 (0-12) levels. The lectical scale adds 4 phases per level from level 6 (about age 2 1/2) through level 12 (the highest adult level we know how to measure). This range is composed of 28 phases. In middle and high school most students perform within the range bounded by the last level of the third tier and the third level of the 4th tier. DiscoTests are designed to capture these 4 levels, which are comprised of 15 phases. The levels are 8, 9, 10 and 11. The phases are 8:1, 8:2, 8:3, 8:4, 9:1...11:4.

We use numbers to represent the levels and phases rather than names, primarily because most of the names that have been used to describe the levels seem to create more confusion than clarity. If you are content to allow your understanding of the levels to gradually emerge as you use DiscoTests, that's fine, but if you want to know more about the scale and its history, please visit our information site, lectica.org. There you will find information about the scoring system and some of the learning sequences we have described.

All DiscoTests are calibrated to the same scale, called the Lectical™* scale. What this means is that a given score, say a score of 10:1, means the same thing on a physics assessment as it does on a social studies assessment. This makes it easier for teachers and parents to compare students' growth in different subjects.

Most people struggle with the idea that there can be one scale. After all, it is clear that subjects like physics and social studies don't have much in common when it comes to subject matter. But, as it turns out, some aspects of the way people think look the same no matter what we're thinking about; these are related to the complexity of our thinking. In fact, the complexity with which people think increases in measurable ways as they become more knowledgeable. This is not to say that a given student will think the same way in all subjects. As a psychologist, I think in a very complex way about psychology, but I think in a much less complex way about chemistry. The same is true for children, who usually develop more quickly in some areas than they do in others.

Although measures of the property of development captured by the Lectical scale have been around for some time, there have been two major roadblocks to using them in assessments. First, early measures were expensive and inefficient, making them impractical for large scale or classroom applications. Second, the distinctions researchers could make between performances were not very fine, which made their measurements impractical for tracking the growth of individual students over time.

A new system, called the Lectical™ Assessment System (LAS), has been developed to solve these problems. It captures distinctions between performances that are more fine-grained than those captured by earlier systems, and it is used as part of a methodology that allows us to study how students learn particular subject matter over time. Such studies provide detailed information about student learning that we convert into practical coding rubrics for teachers. Teachers' codes are translated into Lectical scores with a simple algorithm.

BTW, it is not appropriate to translate lectical scores into A's, B's, and C's. The lectical scale is not a grading system. It is a scale for determining what a student can most benefit from learning next.

 

*Our scale is based on the dynamic skill scale described by Dr. Kurt Fischer of Harvard's Graduate School of Education and its scoring rules were influenced, in part, by Michael Commons' General Stage Scoring System.