In this article, we present research that permits, for the first time, assessing the progress of systemic reforms. The model we discuss identifies critical components of such reform efforts and empirical ways of measuring them.
What Is Systemic Reform?
reform is sometimes called the “third wave” of educational reform, following
higher educational requirements in the 1980s and school restructuring in the
early 1990s. Most states and many
school districts have educational reforms under way that aim at the whole
system. The National Science Foundation early on made the concept widely known
when it launched its Statewide Systemic Initiative program in 1991, followed by
similar programs for large urban districts and consortia of smaller districts.
Other federal agencies have followed suit, for example, the Department of
Education with Goals 2000. By now, the term has entered the vocabulary of
wide acceptance of systemic reform means that a powerful lesson has been
learned: education in grades K-12 will improve only if all the critical
components are addressed in concert. This attitudinal change in itself
represents an irreversible step forward. But what exactly is systemic reform?
And how will we know it when we see it—how can we assess whether it really is
in place and whether it is effective?
National Institute for Science Education has studied these questions for over
three years and held two national Forums to debate them. The central thesis of
systemic reform is that greater coherence (or alignment) of policies and
practices that influence classroom instruction is the only way to create large
numbers of effective schools. The pertinent policies include
for what is to be learned,
based on those standards,
inservice and preservice teacher education,
site autonomy and restructuring,
services from districts and states, and
and professional support.
achieving coherence and high standards among all these components is a complex
undertaking, given our decentralized education system. The complexity can easily
overwhelm reformers, participants in reform, and evaluators alike. For example,
at one of the Forums, systemic reform was described as fluid, nonlinear, context
dependent, cumulative, and slow to develop, given limited resources and
political disputes. This suggests a compelling need for a way to simplify
concepts, action, and evaluation in order to gain a better understanding of the
patterns that exist in systemic reform—what works and why.
A Model of Systemic Reform
Institute is addressing this problem. Clune, who co-directs an Institute team on
systemic reform, has proposed the following model in an Institute monograph:
Reform Initiative (SR), through its purposeful activities, leads to
Systemic Policy (SP), which leads to
rigorously implemented, Standards-based
Curriculum (SC) for all students, which leads to
high Student Achievement (SA) in the
curriculum as taught.
of these variables can be rated along two dimensions: breadth and depth. For
example, if a state concentrated on thorough reform of the mathematics
curriculum in grades 6-8 in its urban middle schools, this likely would
represent a high rating for depth (say, 3 on a scale of 0-5) in curriculum (SC)
but a low rating in breadth (say, 1). Conversely, a reform introducing hands-on
science modules in grades K-12 throughout a state’s schools would likely rate
high on breadth (4) in curriculum but low on depth (1). The Clune monograph
provides a detailed matrix specifying the rating system for each variable, the
components of each variable, and the criteria for rating breadth and depth for
To test the model, Clune and his team rated nine
states on the breadth and depth of their systemic reform initiatives in
mathematics and science education, using information developed by researchers at
SRI International. This quantitative analysis confirmed the relationships
posited in the model: increased Student
Achievement was associated with higher ratings for other variables,
particularly Systemic Reform and Systemic
How does this model help us understand systemic
reform? First and foremost, it has brought clarity to what has been an elusive
concept for those charged with the implementation of systemic reform and the
evaluation of its effects. Formulating four key variables and operationalizing
them through a rating system of two dimensions for each variable has established
a common language for reformers and evaluators. People can no longer argue past
each other as to what is or is not systemic reform, and just how its effects
should be assessed.
Second, analyses using the model and associated
rating scheme allow comparison of reform sites (states, districts, or regional
consortia) despite variations in their reform strategies, such as introducing
new curriculum materials versus training teachers in more effective
instructional strategies, just how they assess student achievement, or how they
garner public support. For example,
the ratings of the nine states showed that seven had made reasonable progress,
four of them considerably so, given the limited time and resources available to
Third, the model makes possible judgments about the
extent of implementation of the different components of systemic reform. For
example, in the analysis of the nine states, higher ratings for breadth and
depth of the reform effort (SR) corresponded to higher ratings for breadth and
depth of policy alignment (SP). But it also was clear that SR and SP were more
fully implemented in most states than curriculum reform (SC), and the effect on student
achievement (SA) was apparent in only three states and generally not well
documented. This probably is to be expected, given the sequence of reform and
the sheer difficulty of making an impact on teachers and students. Perhaps it
also reflects the difficulty and expense of student testing that is tailored to
the needs of evaluation and can capture the more comprehensive student
achievement goals of systemic reform.
Fourth, the quantitative ratings set the stage for
informed qualitative generalizations about the attributes of more and less
successful reforms. In the sample of nine states, the more successful ones had
agency with autonomy but with strong connections to policy makers and
networks of reformers and professional organizations,
state assessments or curriculum controls,
standards-based professional development,
on incentives and resources, and
building school capacity statewide.
Also, the stronger reforms built on a history of prior reform, suggesting that five years is insufficient for building systemic reform from scratch, and that states lacking a base of prior reform should be recognized for building one.
Institute model, which pictures systemic reform as a continuous process with
feedback loops, reminds us that the educational system is dynamic as elements
change for educational, political, or financial reasons. We also need to
remember that reform likely will proceed incrementally. As the reform effort
strengthens, it will lead to gradually stronger policies, making for a gradually
stronger curriculum for more students, which will in due course lead to greater
gains in student achievement.
article was drawn from the Institute’s Research Monograph No. 16, Workshop
Report No. 4, and a book being written by Clune and colleagues on the analysis
of systemic reform.