ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Norm Webb

July 2006

Some students have disabilities that make their participation in state- and district-wide tests impractical. In many cases, students with disabilities who participate in alternate assessments receive curriculum and instruction that differ significantly from those received by other students. The mainstream tests don’t accurately measure their academic achievement. It’s important to know how well alternate assessments align with the curriculum and instruction provided to students with disabilities.

Special education leaders and policymakers have a tool for gathering evidence of the validity of their states’ assessments. A nationally recognized alignment procedure developed by WCER Senior Scientist Norman Webb, this tool has been applied to Wisconsin’s alternate assessment.

Norman Webb and colleagues Andrew Roach and Stephen Elliott analyzed the degree of alignment between Wisconsin’s Model Academic Standards and the Wisconsin Alternate Assessment (WAA), which assesses the academic performance of students with significant disabilities and offers an alternative to the traditional achievement tests. Specifically, Webb’s study asked: Does the WAA adequately measure the concepts and skill areas represented in Wisconsin’s Model Academic Standards? The study found content-related evidence for the WAA’s validity.

Measuring Alignment

Webb says effective schooling depends on coordinating three components of the educational environment: curriculum, instruction, and assessment. The degree to which these elements work together toward student learning is termed alignment, and that provides the foundation of standards-based education reform.

Roach convened a panel to code and rate the alignment between the WAA items and Wisconsin’s Model Academic Standards. The primary role of the panel members was to:

1. Reach consensus on a depth-of-knowledge level rating (see below) for each objective in the Model Academic Standards;
2. Rate the depth-of-knowledge level for each item on the WAA rating scale; and
3. Identify the one or two objectives from the Model Academic Standards corresponding to each WAA item.

Four levels were used to rate depth of knowledge:

• Level 1 (recall) requires simple recall of such information as a fact, definition, term, or simple procedure.
• Level 2 (skill/concept) involves some mental skills, concepts, or processing beyond a habitual response; students must make some decisions about how to approach a problem or activity. Keywords distinguishing a Level 2 item include classify, organize, estimate, collect data, and compare data.
• Level 3 (strategic thinking) requires reasoning, planning, using evidence, and thinking at a higher level.
• Level 4 (extended thinking) requires complex reasoning, planning, developing, and thinking, most likely over an extended time. Cognitive demands are high, and students are required to make connections both within and among subject domains.

Why Alignment is Important

Strengthening the correspondence between students’ individualized education programs (IEPs) and the state’s alternate assessment and academic standards helps ensure that students with significant disabilities are included in a meaningful way in instructional improvement efforts and standards-based reform.

Roach’s panel found that the WAA generally met the criteria for acceptable alignment between assessments and curricular expectations as articulated in academic content standards. By establishing the alignment and curricular relevance of the WAA, Webb’s study provided evidence of the validity of the WAA results as a measure of students’ mastery of the academic concepts and skills outlined in the standards. The investigation also demonstrated the use of a formal procedure—the convened panel—to establish the alignment of an alternate assessment.

To improve the validity and utility of WAA results, the panel did recommend adding items to the WAA Science scale to improve its alignment with the state’s academic standards and to the IEPs and classroom curricula of students with significant disabilities. Three new items were added to the WAA Science scale before the initial WAA implementation year.

This study speaks to the role of sequential development and expert review in promoting the alignment between the policy elements of curriculum, instruction, and assessment systems. Sequential development involves creating and accepting one policy element and then using it as a blueprint for additional policy elements. For more information about alignment, see this report (PDF).

To try the Web-based alignment tool, click here.

For more information about the technical aspects of the evaluation study, see “Issues Related to Judging the Alignment of Curriculum Standards and Assessments” here.