ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Chris Thorn

Rob Meyer

December 2008

WCER’s Value-Added Research Center (VARC) develops, applies, and disseminates value-added and longitudinal research methods for evaluating the performance and effectiveness of teachers, schools, and educational programs and policies. Directed by Rob Meyer, the staff of scientists, researchers, and education professionals apply cross-disciplinary expertise to ground-breaking work on value-added systems and evaluation models. Researcher Chris Thorn gives us an update.

Research Highlights: VARC recently began providing technical assistance to recipients of Teacher Incentive Fund grants from the U.S. Department of Education.

Thorn:  Right. There are 34 grantees across the U.S. They get a total of $95 million annually to support pay for performance systems. We’re collaborating with Vanderbilt University to provide technical assistance to the Department of Education and to individual grantees as they develop and deploy their programs. We’re also working with staff from the Consortium for Policy Research in Education (CPRE) to provide support for teacher and principal evaluation. That’s a major component of pay for performance systems.

Research Highlights: You’re continuing to develop and implement the value-added model.

Thorn: We now have three years of test data, and down to the classroom level. We’ve developed production value-added models for Madison Metropolitan School District, Milwaukee Public Schools, and Chicago Public Schools. We’re also doing a prototype model for Wisconsin’s Department of Public Instruction (DPI) to demonstrate the use of value-added at the state level. This work will compare the use of state data at different district sizes, using data from Milwaukee, Madison, and CESA 2 (a cooperative educational services agency).

There’s also a research component to this work. We traditionally develop complex diagnostics and model refinements to test whether simplifying assumptions in the model are acceptable. This is something we do in house and then share with district staff. The implications of violating model assumptions are really their call to interpret. This level of engagement is what drives the research model. There are real concerns about how best to handle mobile students and teachers, different testing windows, etc.

Research Highlights: What is the project doing with data structures, data systems, and data quality?

Thorn: Many districts and states find that their data systems might be adequate for annual compliance reporting, but they’re not up to the task of real longitudinal assessment for students and adults. We work with staff and vendors at State Education Agencies and Local Educational Agencies to examine the technical tools and human process used to collect, manage, analyze, and report out information. Often, the primary task is making clear how the requirements of longitudinal analysis test assumptions of data quality across all information systems. We share solutions deployed in other settings and work with local teams do develop fixes for unique contexts.

Research Highlights: What’s going on in Milwaukee?

Thorn: We've been working in Milwaukee on a variety of evaluation projects to determine the effectiveness of several programs. We’re about to launch a new project that would require senior leadership to develop an "evaluation impact statement" for every major initiative. All interventions would be evaluated for effectiveness. They would also be deployed in ways that maximize the district's ability to judge effectiveness, including random assignment studies.

Research Highlights: Professional development is also an important part of what you do.

Thorn: Yes. We’re helping Milwaukee build a professional development (PD) cadre. We’ve developed a core PD curriculum for a train-the-trainer model. This curriculum includes training about the uses of assessment data and it compares and contrasts the different uses for attainment and value-added outcome measures. We’re likely to begin similar work in Chicago.

WCER’s Value-Added Research Center receives funding from several sources including the U.S. Department of Education’s Institute of Education Science, National Center for Education Statistics, Joyce Foundation, Spencer Foundation, and the Chicago Community Trust. Research partners include the Wisconsin Department of Public Instruction, MilwaukeePublic School System, Chicago Public Schools, Dallas Independent Schools, Minneapolis Public Schools and Teacher Incentive Fund grant recipients.