ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Andrew Porter

Teachers get lots of advice and support from a variety of sources about what to teach. But do they really teach what is described in content standards? Do they teach what is in the textbook? Do they teach what is tested?

Classroom teachers are the ultimate arbiters of what is taught, and how. Regardless of what a state policy requires or what a district curriculum spells out, the classroom teacher ultimately decides how much time to allocate to particular school subjects, what topics to cover, when and in what order, to what standards of achievement, and to which students. Collectively, teachers' decisions, and their implementation, define the content of instruction.

Knowing what teachers actually teach is important to educators and policymakers who need to determine whether and to what degree there is overlap (or alignment) between what is taught, what is tested, and what national, state, and local content standards prescribe.

Tools for measuring content and alignment

For the past 25 years, Andrew Porter and his colleagues have studied teachers' content decision making in mathematics and science. He has developed three kinds of tools for measuring content and alignment:

1. Teacher surveys describing the content of instruction
2. Content analyses of instructional materials, including assessments
3. Indices of alignment between instructional content, instructional materials, and standards

The power of these tools lies in the uniform language they use for describing content. It is this uniform language that makes it possible to build indices of alignment.

The language developed by Porter and colleagues consists of uniform descriptors of topics (level of coverage) and student expectations (categories of cognitive demand). The level of coverage and the categories of cognitive demand form the columns and rows of a two-dimensional matrix. The content of instruction is described at the intersection between level of coverage and cognitive demand.

The values placed in each cell of the matrix reflect data gathered from teacher surveys or from content analyses of instructional materials. For example, the surveys ask teachers to indicate, for the past school year,

1. the amount of time they devoted to each topic (the level of coverage); and
2. for each topic, the relative emphasis they gave to each student expectation (category of cognitive demand).

Porter and colleagues then analyze the results from surveys and content analyses to produce topographical maps that graphically display the content of instruction.

Porter's content matrix can also be used as a tool for developing content standards. The power of the content matrix in this context is that it facilitates clarity in making hard choices, Porter says. State content standards-and even national-level standards, like the mathematics standards developed by the National Council of Teachers of Mathematics (NCTM)-tend to include much more content than can be taught in depth. Educators and policymakers can use the content matrix (and the topographical maps displaying the content) to build standards that clearly define not only what is to be taught but also what is not to be taught.

Porter's method of measuring the content of instruction and alignment differs from other approaches in two ways:

1. The tools allow independent and replicable descriptions of the content of instructional practice and instructional materials. The uniform language for measuring content ensures descriptions at a consistent level of depth and specificity.
2. The uniform language allows alignment to be measured across a large number of instructional materials and instructional practices.

Using the tools to study alignment

Most approaches to alignment of assessments with standards start with a particular state's standards and ask: To what extent does the content in those standards appear on the test? Such analyses are unique to each state. They don't allow comparisons between states or comparisons between state and other professional standards. But the uniform content language developed by Porter and colleagues allows one to compare alignment between states, and to national standards like the NCTM standards.

In a study done at the American Institutes for Research, Rebecca Herman and Laura Desimone recently used Porter's tools to study the alignment of standards with assessments in four states. Their data showed that the assessment of each state was no more aligned to its own standards than to the standards of the other states or to those of NCTM. Porter says that perhaps the state standards are not sufficiently specific to allow an assessment to be tightly aligned with them. But he says a more likely possibility is that states have much more work to do to bring their assessments into alignment with their standards. This finding is one about which U.S. Department of Education officials are expressing concern, Porter says.

The tools described here are used for science as well as mathematics. Some early work has also been done in the areas of reading and language arts and social studies. Following are examples of how the tools can be used to describe instructional practices, instructional materials, and alignment.

Describing instructional practices

Good measures of the content of instruction can serve

1. to define the process of teacher decision making in reaction to the various messages that teachers receive about what should be taught;
2. to describe the implemented curriculum or to measure the degree of implementation of a new curriculum;
3. to validate transcript studies; and
4. to provide the basis for powerful professional development experiences.

With regard to professional development, Andrew Porter and colleagues are using their measures of the content of instruction as the core of a new program on teachers' instructional practices, with funding from the National Science Foundation. The program begins by having teachers complete surveys describing the content of their instruction. Porter and colleagues then analyze the results and produce graphic displays that are returned to the teachers. Finally, teacher teams use the data to answer the following questions:

1. Is the content of our instruction what we want it to be? Is it aligned with our tests and content standards?
2. Are the differences in what teachers are teaching appropriate?
3. Do the prerequisite courses provide the content needed for effective grade-to-grade articulation?

In research, indices of alignment between the content of instruction and a student achievement test can be used as a control variable in studies of the effects of pedagogical practices on student achievement gains. An index of alignment can also be used as a descriptive variable in assessing the coherence of a state's or district's curriculum policy system.

For more information see Porter's page at the WCER Web site, www.andyporter.org