Task Module Assessment System
Goal 2: Plan, conceptualize, and develop task module assessment strategies, and examine validity and reliability
Concurrent to the studies described in Goal 1, the TMAS project is conducting a study to begin to address a second question of interest: What are the appropriate assessment systems for students in the gap?
It is not possible to develop an entire assessment system without first identifying the students who will be served by that assessment system, but because of time constraints, the TMAS project has had to begin the process of assessment development without first having the results of its current studies. Building on wider research about effective assessments, the project is exploring one strand of assessment type. The assessment unpacks what students do and do not know around a particular strand of mathematics (in this case algebra), and allows teachers to better understand the extent to which students are able to combine this understanding to perform complex problems (click here for a Sample Assessment Item). Using a web-based system, the assessment can be administered in less than an hour.
The prototype test was designed to measure students’ understanding of the skills associated with the Functions and Algebra Grade-Level Expectations. The prototype test is based on a set of items from the most recent state assessment. The items were selected because they are complex problems that require students to apply multiple skills and concepts. A series of new items that tap into skills related to the original item and corresponding GLE were developed with two purposes in mind. First, items were developed to more deeply probe students’ understanding of the component skills both individually and in combination. Second, alternate representations of items were developed to determine if modifications to the original problem would enable students to solve it. For example, in some cases the context of the problem was modified or removed while for others, the presentation of information was changed (i.e. from vertical to horizontal, from a table to a graph) and in some cases, the problem was simplified (i.e. by using whole numbers rather than fractions, reducing the number of variables, or reducing the number of steps required to solve the problem). Because this assessment module is built directly from the state assessment itself, it does not entail developing an entirely new assessment system, but is a modification of the current system.
The new assessment approach is being pilot tested in February and March. All 8th grade mathematics teachers in Maine, New Hampshire, Rhode Island and Vermont are invited to participate in the study. Even though this strategy was designed to better assess students in the gap, the pilot study will include both students who are in the gap and students who are not. In this early stage of development, it is important to examine how students at all ability levels perform on this type of assessment. More specifically, the purpose of the study is to examine:
- the extent to which items designed to decompose skills and knowledge required to solve complex algebraic problems form a unidimensional scale;
- if so, the extent to which kids in the gap are able to succeed on decomposed items while struggling with the complex items; and
- the extent to which students who perform well on the complex items also perform well on the decomposed items.
In order to participate in the study, teachers must first register online at http://www.bc.edu/tmas. Registration involves answering a few questions and takes only a few minutes. After registering, teachers can administer the computer-based test to their students. The test can be administered any time during February and March. The test is online so immediately after students complete it, teachers can receive feedback on their performance.