ABOUT WCER NEWS Events Feature Stories WCER News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS Active Projects Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN
Intervention Programs Can Have Powerful Effects
Instructional improvement intervention programs like Accelerated Schools, America’s Choice, and Success for All can have powerful effects on the enacted curriculum in U.S. schools.UW-Madison education professor Eric Camburn and colleagues* found that teachers do not vary the curricular content of literacy instruction for students in their classrooms. For 2 of the 3 instructional improvement interventions studied, variation in teachers' instruction suggested that many teachers used practices advocated by the improvement programs.
Camburn and colleagues reached these findings while studying literacy instruction in third-grade classrooms in 53 high-poverty schools. These schools were participating in the 6-year Study of Instructional Improvement.
The study analyzed instructional logs for reading and language arts from more than 150 third-grade teachers over the course of one school year. Teacher logs provide reasonably accurate data about the enacted curriculum when they are filled out immediately after lessons and with enough frequency to discriminate reliably across objects of measurement.
The teachers taught in 33 school districts in 11 states. Fifteen schools were participating in Accelerated Schools, 15 were in America’s Choice, and 16 were in Success for All. The remaining 7 schools were chosen as comparison sites because they were not participating in any of the three reform programs.
Camburn and colleagues examined the logs and the enacted curriculum along two dimensions: For the first, they characterized the curriculum in terms of nine reading/language arts strands. Then, within each strand they identified a second dimension of the curriculum—the developmental level, or the level of difficulty or cognitive demand of the skills being taught.
This approach to measuring the enacted curriculum moves beyond a focus on “the overlap between what is taught and what is tested” to measure the degree and nature of curricular focus found in particular lessons and the level of difficulty of instructional content taught on particular days.
Camburn points out that a researcher’s ability to discriminate across teachers increases as the number of occasions of measurement increases. But this reliability also depends on how much variation in content coverage exists (a) among teachers and (b) for each teacher from one occasion to the next. If teachers vary greatly in cumulative content coverage, but there is little occasion-to-occasion variance for individual teachers, relatively few observations are needed to discriminate among teachers. Conversely, if differences among teachers in cumulative content coverage are smaller and/or occasion variance for individual teachers is larger, relatively more observations are needed.
Several themes emerged from the analyses:
1. The largest amount of variation in the enacted curriculum occurs at the occasion-of-measurement level. This suggests that teachers vary to a significant extent the content and difficulty of the skills they teach from day to day.
2. There is little evidence of differentiation in either the amount or the skill level of reading comprehension or writing instruction students in the same classroom receive over the course of a year.
3. Even with large occasion variance in content and skill coverage, it is possible to discriminate reliably among teachers’ patterns of curriculum enactment.
4. The intervention programs—Accelerated Schools, America’s Choice, and Success for All—had large effects on the enacted curriculum. The effects were consistent with their intended designs.
These findings suggest that teachers have less autonomy in enacting the curriculum than is suggested by popular images of schools as loosely coupled systems and teachers as curriculum brokers. In fact, Camburn says, intervention programs can have powerful effects on the enacted curriculum in U.S. schools. Curriculum coverage in U.S. classrooms can be treated as an alterable variable in discussions of education reform.
This research is one of a number of recent studies Camburn has conducted on the measurement of instruction. The research reported here and a mixed-method study validating measures of instruction from a daily log appear in a special issue of The Elementary School Journal, Vol. 105, No. 1, September 2004. A third paper assessing the validity of measures of instruction based on annual surveys was presented at AERA this spring.
* Camburn’s colleagues in this study were Brian Rowan and Richard Correnti at the University of Michigan.Funding: Grants to the Consortium for Policy Research in Education from the Atlantic Philanthropies (USA), the National Science Foundation, and the U.S. Department of Education.