ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Norman Webb & Iris Weiss

April 2004

Students are learning more challenging mathematics and science content, thanks to the National Science Foundation's Statewide Systemic Initiatives (SSI) program.

A team headed by WCER researcher Norman Webb and Horizon Research, Inc. colleague Iris Weiss recently concluded their study of the SSI program, which began in the early 1990s. The pilot program enabled 25 states and Puerto Rico to help students improve their learning of challenging mathematics and science content by making simultaneous changes in many areas of education-for example, organizational structure and decision making, curriculum content and learning goals, facilities and equipment, and assessment of student achievement.

Webb and Weiss evaluated the impact of SSI on student achievement and the lessons that could be learned from implementing such a program. They found that student achievement varied across states with SSI awards, apparently due to factors such as prior conditions in education and accountability systems and the duration of NSF funding.

The study reached three main conclusions-technical dimensions of developing capacity and maintaining quality, political dimensions of developing guidance and incentives, and a interactions with funders. An example from each area included:

1. A technical lesson: It was vital to build flexibility into the SSI program so that information produced by research, evaluation, and monitoring could be effectively used.
2. A political lesson: Creating partnerships with policy organizations helped establish a supportive context for reform.
3. A lesson on interactions with funders: SSI researchers and funders needed to develop a shared understanding of the reform strategies as they fit local contexts.

The study's findings generally agree with theories of systemic reform. For example, the effectiveness of state assessments and accountability policies appears to be a strong factor in improved student performance. State policies aligned with the goals of a systemic initiative can result in substantial gains in student achievement in a relatively short time when accompanied by a strong statewide infrastructure that supports teachers and schools as they change their practices.

Multiple components in concert

More specifically, Webb and Weiss found that:

1. Statewide student achievement gains over 4 years were more likely when reform efforts targeted state policy as much as, or more than, teachers' classroom practice.
2. Statewide assessment policies and practices constituted important components of systemic reform. The existence of a state assessment program seemed to be related to statewide achievement gains, particularly when criterion-referenced tests were used.
3. When state policies did not support SSI goals, reform efforts appeared to be compromised or even undermined.
4. When assessments were aligned with SSI goals, reform-related instructional practices increased. When they were not aligned, reform-related instructional practices decreased or did not change.
5. In all SSI states, the alignment of state policies and assessments with the SSI goals appeared to be an important influence on statewide student achievement.

The close fit between improved performance and SSI funding (in the context of state support for SSI goals) suggests that change is most effective when multiple components are addressed in concert. The SSIs had the most effect when they served as catalysts for other state reform efforts.

Reform can result in substantial achievement gains in a relatively short time when state policies are aligned with the goals of a systemic initiative, and when state infrastructure supports teachers and schools as they change their practices.

Although it is important to consider the technical issues concerning a program's functioning, Webb says, it's also essential to address state political decisions and consider negotiations with the funder. It takes all elements to garner the support necessary to sustain an effort long enough to achieve a measurable improvement in student learning.

To read the entire report, see the Technical Reports page of Webb's site:
http://facstaff.wcer.wisc.edu/normw/technical_reports%20Page%201.htm

Iris Weiss can be reached at Horizon Research, www.horizon-research.com