|
Analyzing Instructional Content
September 2004
Students perform best on tests that assess subject matter content students have had the opportunity to learn. Though supported time and again by research*, this observation is little more than common sense. Until recently, however, educators have had little ability to examine the relationship between the content of assessments and the content of instruction. New developments in survey instrumentation and procedures for conducting content analyses of assessments and standards are making it easier for researchers, teachers, and others to look at the relationships between instruction, assessments, and content standards in a more detailed way than ever before.
The tools developed by the Surveys of Enacted Curriculum (SEC) project offer a practical way to collect, report, and analyze data about what, and how, teachers teach. This data makes it possible to compare one school's patterns of content coverage and classroom practice to district and state standards and assessment results.
Evaluating and improving curriculum in language arts, reading, math, and science require reliable, comparable data on the degree of consistency in subject content taught and classroom practices used. SEC offers a quantitative method for analyzing instructional content and practices that provides data for making decisions on improving curriculum and instruction. The surveys help educators
- align curriculum, instruction, and assessment;
- track school performance against standards;
- monitor changes in instructional practice at the school and their effects on student performance; and
- identify professional development needs.
SEC engages teachers in honest, open, and objective dialogue about teaching. SEC supports reading and English language arts, mathematics, and science-the subjects targeted by the No Child Left Behind Act.
SEC was developed by a collaborative of state education specialists and WCER researchers. The collaborative is led by Rolf Blank, director of education indicators at the Council of Chief State School Officers (CCSSO). Much of the survey design and content is based on research conducted by former WCER Director Andrew Porter (now at Vanderbilt University) and WCER Researcher John Smithson.
The enacted curriculum surveys provide teachers a detailed set of indicators that support reflection on their instructional strategies and can inform their curriculum planning (see illustration). When combined with content analyses of state standards and assessments, the SEC data provide an efficient and highly quantitative means for calculating alignment measures that can be used by researchers and others to examine issues of alignment of instruction to assessments and standards, as well as the alignment of assessments to standards.
Uses
Administrators and instructional resource staff use SEC results to plan professional development, monitor progress toward local and state curriculum reform goals, and initiate conversations among teachers about issues related to strengthening curriculum and instruction. By using the SEC Online web site (www.seconline.org), teachers complete the surveys on-line, review and reflect on their own results, and compare their results to their school, district and state.
The survey data can serve as a basis for workshops focused on curriculum and instruction. A compact disc contains several professional development activity guides and materials designed to engage teachers and other education professionals in activities related to curriculum and instruction.
SEC data provide policymakers a tool for collecting information about classroom practice on a scale not possible through observation or interview methods. Such large-scale data provide a means for establishing a baseline and monitoring progress on goals related to curriculum and instruction. Educators can then examine the effects of policy tools and initiatives.
Researchers can use the SEC tools to analyze the effects of instruction on student achievement. Additionally, by providing a means for holding instruction constant in statistical analyses, SEC helps researchers to better examine the contribution of competing pedagogies to student achievement gains.
Scenarios
Following are three scenarios illustrating the ways schools can use SEC:
- To align curriculum with assessments: Sandy Long, the principal of Shady Tree Elementary, wants to launch an initiative to more closely align what is taught in the classroom to the state standards and assessments. She uses SEC to provide a baseline measure of her teachers' instructional practices in English/language arts. She'll identify gaps and institute changes in the curriculum. She plans to work with her staff to administer SEC again next year to assess progress.
- To align algebra curricula across multiple courses: Teachers at Mountville High School have been asked to improve the alignment between algebra courses. They decide to use SEC to look at Pre-Algebra, Algebra I, and Algebra II. With the results from their surveys in front of them, the teachers begin to have conversations about what changes need to be made to make sure students completing one course are prepared for the next. After talking about the curriculum, the teachers also look at how different practices in the classroom affect student learning.
- To improve professional development in science: Connersville School District has hired Maria Thompson, a professional development consultant, to explore what professional development is needed by district middle school staff, particularly in the area of science. As a starting point, Maria administers the SEC science content and practice surveys to the teachers to assess their professional development needs and wants. In addition to helping identify professional development needs, the SEC process will create a constructive and open conversation about professional development among school staff.
Smithson says that using SEC data allows educators to work smarter, not harder. "Many schools are demanding that teachers examine assessment results and identify ways to raise scores," Smithson says, "but those results provide teachers only very limited information about the nature of the assessed content. The SEC data provide an important tool for assisting teachers in making decisions about what content to cover, and how to cover it."
For further reading:
* Gamoran, A., Porter, A.C., Smithson, J., & White, P.A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19(4), 325-338.
McKnight, C.C., Crosswhite, F.J., Dossey, J.A., Kifer, E., Swofford, J.O., Travers, K.J., et al. (1987). The underachieving curriculum: Assessing U.S. school mathematics from an international perspective. Champaign, IL: Stipes.
Rowan, B. (1998). The task characteristics of, teaching: Implications for the organizational design of schools. In R. Bernhardt, C.N. Hedley, G. Cattaro, & V. Svolopoulos (Eds.), Curriculum leadership: Rethinking schools for the 21st century. Cresskill, NJ: Hampton Press.
More information online:
Support for SEC development was provided by member states, the National Science Foundation, Learning Point Associates, and the U.S. Department of Education.
|
|
