ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

How can a large urban school district advance systemwide change?

In a study of systemic reform in the Milwaukee Public Schools, UW-Madison Professor William Clune and Senior Scientist Norman Webb use an approach they call embedded research.

Embedded research integrates the best of traditional approaches: objectivity and subjectivity, technical assistance and evaluation, and qualitative research and quantitative research.

Embedded research looks at existing elements of the school system from the standpoint of systemic theory, but within the practical constraints of district culture and operations. The desired result is to facilitate district actions towards an improved school system configuration. This research began in WCER’s National Institute for Science Education with funding from the National Science Foundation. It continues now with funding from the Joyce Foundation and the Helen Bader Foundation.

Clune and Webb hope that their findings will serve as a detailed case study of systemic change. The potential to generalize from these findings will depend on the team’s success in identifying the path of system change and how the complex array of components common to any large urban district interact to further or retard this change.

Rather than just doing a study on the district, Clune and Webb have been working with the district. Their purposes and perspectives coincide with design experiment, but separate their work from experimental research.

WhileClune and Webb’s research shares some aspects of design experiments, their approach differs in n the magnitude of the study. Design experiments, as developed by Ann Brown (1992), implemented cognitive learning theory in a classroom setting. In 1998, MPS was the nation’s 15th largest school district, with about 100,00 students enrolled in more than 150 schools. Clune says, “For us as researchers to even assume we could implement or design an implementation intervention for the entire Milwaukee school district would be simply be naïve.”

Clune and Webb’s current research focuses on helping the district in three areas.

1. Developing a revised accountability and assessment system that aligns with state and district standards, promotion and graduation requirements, and utilizes a value-added analysis approach.
2. Creating methods by which individual schools can acquire district data and utilize the data for school improvement and increasing student achievement.
3. Collaborating with district staff on developing the proposed system of classroom-based assessments, which will serve as a vital component of a balanced system of assessment of student achievement in an era of increased importance of standardized tests.

Clune has developed a model of embedded research as systemic capacity building. In the forthcoming book, Theory and Practice of Systemic Reform of Mathematics and Science Education, he develops a model with five components: a theoretical base, inputs, building understanding, outputs, and practical feasibility (see illustration).

“Our embedded research with Milwaukee Public Schools is guided by a theory of systemic reform,” Clune says. “The theory requires the alignment of system components such as standards and assessment. The research also incorporates system and school accountability and assessment principles.”

Embedded research, as applied in the Milwaukee Public Schools, gives researchers like Clune and Webb a deeper understanding of the district’s operations than if they had remained external observers and data gatherers. Clune says extended interaction with district staff in rethinking the assessment system has helped his team better understand how key ideas are interpreted. Understanding how district staff members use terms, such as ‘value-added’ and ‘multiple measures’ as well as ‘standards’ and ‘alignment,’ has required numerous conversations. But the effort pays off as researchers come to better understand the underlying district thinking.

The form the new assessment system will assume and the criteria used for judging graduation and promotion requirements will be important indications of the effect of the embedded research approach in the district. “We do believe district staff members are thinking more deeply about value-added analysis and multiple measures,” says fellow researcher Norman Webb, “but other pressures, cost and manageability, may be of greater concern.” Despite the limitations imposed by these district realities, the intermediate research goals certainly are being realized: helping the district think systemically through some of its most pressing problems.

Clune and Webb have a multidisciplinary research team, with expertise in law, policy, statistics, psychometrics, evaluation, student assessment, data systems, school improvement, professional development, and special education. Drawing on these multiple perspectives facilitates an objective interaction and feedback between research studies. This collaboration enhances the learning and improves the team’s ability to respond to the district and to offer meaningful technical assistance.

For more information contact project researcher Sarah Mason at samason@facstaff.wisc.edu.