ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Taking Reform to Scale

February 2007

Geoffrey Borman

The federal government’s continued support of replicable, research-based educational programs, funded through Title I and other national policies, could revolutionize school improvement in some of the most challenging contexts, says UW-Madison education professor Geoffrey Borman. Across varying contexts, Direct Instruction, the Comer School Development Program, and Success for All have shown robust results and have shown that, in general, they can be expected to improve students’ test scores. These three models stand out from other available comprehensive school reform (CSR) designs by the quantity and generalizability of their outcomes, the reliable positive effects on student achievement, and the overall quality of the evidence.

Borman says perhaps the most striking theme of his recent study of Title I and CSR results is the wide variety in overall effects of these national efforts to bring reform to high-poverty schools. Each of the 29 CSR programs offers a distinctive and replicable model of school reform, but the successes schools experience depend upon how well they “buy into” and sustain the tenets of the various approaches. In other words, implementation matters.

Comprehensive School Reform (CSR) means using proven strategies to improve the school as a whole, instead of piecemeal reforms that are common in education. CSR builds on state and local efforts to connect higher standards and school improvement by helping to fund the initial implementation of reforms that are coordinated with, and sustained by, all the resources available to the school, including federal, state, local and private resources. The goal is to enable all children, including children from low-income families, children with limited English proficiency, and children with disabilities, to reach challenging academic standards.

Borman’s meta-analysis did find limitations in quality and quantity of studies supporting the effects of CSR and Title I. For example, despite annual governmental expenditures of approximately $10 billion and a history of nearly 40 years, Title I itself has never been subjected to randomized trials. There have been large-scale evaluations of Title I, but they typically have provided nationally representative survey data describing the characteristics of Title I and non-Title I schools, the characteristics of Title I and non-Title I students, and the achievement outcomes of participants and non-participants.

Quasi-experimental comparisons of outcomes among Title I and non-Title I students have provided some insights into the potential achievement effects of the program, Borman says. But he urges researchers to spend less time attempting to generate national estimates of the program’s characteristics and effectiveness, and more time on studying the effectiveness of specific interventions that could be funded under Title I.

When it comes to implementing clear and replicable strategies for school change, policy mandates—even with local flexibility—are less likely to produce educational reform and improved student achievement than provider-based assistance. Although his study found that clearer federal mandates were associated with improved implementation and effects of Title I, these efforts produced only modest effects on student achievement outcomes.

In contrast, the most successful CSR models have enjoyed sustained periods of development, evaluation, and refinement. They provide clear and replicable strategies for reforming schools. Despite being known as “comprehensive” models, Success for All, Direct Instruction, and the Comer School Development Program focus on improvement in one core area: literacy instruction. These clear, focused, and well-supported school-based models of improvement are in stark contrast to top-down direction and flexibility for educational reform.

The results from national efforts suggest that large-scale reform can produce widespread, but modest, achievement effects. But better evidence is needed to provide both summative and formative appraisals of current and future national efforts to scale up reform in high-poverty schools. Rather than approving CSR programs on the basis of the 11 federal requirements (e.g., parent outreach program, clear goals and benchmarks) that make a model “comprehensive,” schools and policymakers should pay even stronger attention to model outputs.

Adapted from “National Efforts to Bring Reform to Scale in High-Poverty Schools: Outcomes and Implications,” Review of Research in Education (29) 2005, pp. 1-27).