Vol. 2, No. 3. - August 1999

Measuring Progress Toward Equity in Science and Mathematics Education

By Jane Butler Kahle

October 1997 marked the 40th anniversary of Sputnik, the rocket that provided an early impetus for reform of science and mathematics education in the United States. Forty years and several permutations later, we are still involved in reform. Now the focus is on making reform systemic and enabling all students to gain literacy in mathematics, technology, and science, rather than educating relatively few to become mathematicians, engineers, and scientists.

The reform of the 1960s did not address the interests or needs of many students who, by nature of their culture, gender, or physical or economic condition, were less attuned to, or had less access to, quality science and mathematics education. Rather, classes were tracked and only a few students benefited. In the ensuing 40 years, the numbers of those historically excluded students increased dramatically.

The driving force behind the current reform movement is the need to remain economically, scientifically, and technologically competitive with other developed nations. Increasingly, as K–12 students have become more diverse and as the underrepresentation of whole groups of students in science and mathematics has become more visible, we have come to understand that this time the reform of science and mathematics education must be both systemic and equitable. That is, the reform must address multiple parts of an educational system, and it must increase the access, retention, and achievement of students from all subgroups in high quality science and mathematics programs. Curricula must change to represent varied interests, to implement more effective ways of organizing classrooms and schools and of providing instruction, and to use assessments that include multiple ways of demonstrating learning and competencies. In addition, policies that determine both the quantity of courses and the quality of the educational experience (e.g., teacher qualifications, teaching resources, and academic tracking) must be reviewed and changed to ensure equitable reform. As our student population becomes ever more diverse, simple and defensible ways to measure progress toward meeting the needs and expectations of all students have become increasingly important. Equity, or high quality science and mathematics education for all students, matters in today’s reform.

One way to approach these issues is to take stock and assess where a system is along a continuum toward equity in reform. Each system, defined as a school district in this discussion (but, conceptually, a system may be any educational unit—from an individual class to an entire state), needs to identify guideposts along the path to high quality education in science and mathematics for all students. Taken together, those guideposts form an equity metric, a way to measure progress toward equity.

This Brief proposes and describes a methodology for developing and using equity metrics in ways that measure genuine progress toward high quality science and mathematics education for all students.

Developing an Equity Metric: From Guideposts to Indicators

Guideposts for equity may be found in the analysis of large national and international databases, in research literature, and in the changing policies and practices of the current reforms of science and mathematics education. For purposes of monitoring a system’s progress toward equity, it is important to provide easily understood and acceptable data. Therefore, only measurable guideposts, commonly called indicators, are included in this discussion of equity metrics. These indicators are drawn from three large databases (NELS:88, High School and Beyond, and TIMSS),[i] NSF’s indicators of quality mathematics and science education (National Science Foundation, 1996), and the research literature for evidence of inequality in access, retention, and/or achievement across student subgroups. If evidence of inequity on a type of indicator was found in two or more sources (e.g., unequal enrollments by subgroups in eighth-grade algebra), the indicator has been included in the metric.

Next, the identified indicators have been sorted by grade levels. This helps address two questions:

·          At which grade levels are information about students collected?

·          At which levels are enrollment, participation, and achievement critical for a student’s continued access to and/or progress in science and mathematics?

The sorting suggests leverage points in the educational system that are related to critical times in a child’s education; that is, periods when educational systems routinely gather data concerning specific placement (e.g., general mathematics or algebra) and performance (e.g., standard achievement tests, high school graduation). The leverage points identified here are preschool and  fourth, eighth, tenth, and twelfth grades. Indicators have been sorted by appropriate leverage points.

Lastly, indicators of general reform were identified. Using the above criteria and databases, indicators of systemwide progress have been added to the metric, shown in Figure 1.

  A primary reason for caution when using this approach is that gender differences may not be identified. Girls and boys enroll in equal numbers in algebra, biology, calculus, chemistry, and trigonometry. Further, on average, girls achieve higher grades in those courses than boys do. However, the enrollment patterns in physics are not equal, suggesting that neither course enrollment patterns nor achievement levels in science and mathematics predict girls’ enrollment in physics.[ii]

  Once indicators have been identified, an educational system can select among them to design its own equity metric. The indicators included in the model equity metric in Figure 1 have been selected to meet the following criteria:

·    They are sensitive to diversity among subgroups of students, teachers, and others.

·    They can be used to inform action, not just to define the present state.

·    They are flexible, because not all metrics are relevant to all parts of the system.

·    They distinguish among access, retention, and achievement.

·    They are directed toward leverage points in the system.

·    They are feasible to use (i.e., affordable).

Constructing a Metric: Selecting Indicators

Indicators may vary across time, changing to address different factors and/or conditions. For example, early studies suggested that  teacher qualifications were an indicator of inequity, as they differed between schools serving primarily minority students and those enrolling primarily majority students. However, analysis of current databases indicates that teachers of minority students are not necessarily less well prepared than teachers of majority students in terms of certification, number of years in teaching, or educational level. There are no significant differences on these indicators in science, and the only difference in mathematics is in the percentage of certified teachers of Native American students compared to all other groups. Therefore, instead of using certification, experience, and attainment of a bachelor’s degree as indicators of inequity in teacher qualifications, indicators of the quality of the teacher preparation and professionalization programs may be needed. For example, more useful indicators may include number of credits in science and mathematics courses, evidence of advanced as well as introductory science and mathematics courses in the undergraduate program, length and quality of practicum or intern experience, and certification by the National Science Teachers Association or the National Board for Professional Teaching Standards.

Other indicators, such as Home Resources, may be composed of several factors. For example, attendance at preschool has been found to be an indicator of inequity for Hispanic and Native American children, while presence of a table or desk for a student’s own use and presence of a computer in the home differ between minority and majority students and have been linked to student achievement in many of the 41 countries (including the United States) in the Third International Mathematics and Science Study (TIMSS) (Beaton, Martin et al., 1996; Beaton, Mullis et al., 1996). Those components are easy to measure and may be assessed as part of the indicator.

The indicator Student Attitudes and Beliefs addresses the documented decline in positive attitudes in science between fourth and twelfth grades. It is relatively easy to measure and also can be used to address gender equity, because the decline in attitudes is greater for girls than for boys. [iii]

Another indicator, Learning Behavior, includes absenteeism and tardiness (which are easy to measure and indicate degree of student engagement in learning), the priority students place on learning, and the amount of competition students face for grades (increasing competition correlates with decreasing achievement among non-Asian minority groups).

One of the most interesting indicators is Quantity/Quality of Math/Science Courses. Recent studies suggest that to provide equitable education we must move beyond counting the hours or numbers of courses and assuming that courses with similar titles are comparable. Observational studies, teacher logs, teacher and student surveys, and student portfolios are some of the ways by which we can assess the quality of a course. Although indicators of quality (depth of coverage and mode of instruction) are needed, enrollment in key gatekeeping courses (such as eighth-grade algebra or high school geometry) and the Availability of Advanced Math/Science Courses are also critical indicators of high quality mathematics and science education. Other key indicators, found in Figure 1, are both the intent to enroll in an Academic Program in the eighth grade and actual enrollment in one in the tenth grade.

Quality of Professional Development is included as an overall indicator of movement toward equity. Teachers need access to life-long learning and skill development to implement challenging curriculum, to use varied instructional strategies, to include multiple types of authentic assessments in their classrooms, and to improve their understanding of the backgrounds of students from diverse subgroups. Measurement of the quality of teacher professional development needs to move beyond the number of college or continuing education credits accrued toward the quality of outcomes. Evidence of changing practices, behaviors, and attitudes among teachers and students that may be collected through teacher logs, student journals, audio and video tapes, and interviews is needed. Further, a critical indicator of the quality of professional development is improvement in the retention and achievement of students in all subgroups.

Different Challenges, Different Indicators

Once a system has articulated its equity goals and has identified guideposts or indicators of equity, it must formulate a working plan for becoming more equitable, as well as a timeline for initiating components in its plan. It is estimated that systems will need at least five years to demonstrate progress toward equity, using the indicators in Figure 1. Initially, baseline data and appropriate benchmarks of progress must be identified. Next, ways of monitoring progress are needed. Finally, collection and analysis of data, coupled with dissemination and discussion of the findings, must occur. Fortunately, national databases suggest key indicators as well as ones that are applicable for specific student subgroups.

What are key indicators that any system is becoming more equitable? First, retention and achievement in eighth-grade algebra are key indicators of a student’s probability of achieving a high quality education in mathematics and science. Second, although not easily quantified, the quality of the content of science and mathematics courses is critical. Third, a clear indication of progress is provided by data from achievement tests that show narrowing of gaps concomitant with increased achievement by all subgroups of students. Fourth, evidence that teaching practices are changing in ways that involve students actively in learning is important, because active engagement enhances both interest and achievement levels of students who historically have been underrepresented in science and mathematics (Stevens, 1996). Although it is tempting to continue to identify key indicators, these four will indicate movement toward equity and provide salient guideposts along the way.

Another approach is to look for indicators that address a given system’s priorities. In a rural school system where children have similar ethnic/racial backgrounds and speak English at home, movement toward equity may involve removing differences between girls and boys. What are key indicators of gender equity? First, given that girls exhibit a greater decline than boys in attitudes about, and interest in, science, a key indicator of gender equity is sustained positive attitudes and interest levels as girls proceed from fourth grade (where girls are as positive about science and as interested as boys are) through high school. Second, evidence of cooperative learning groups, of activities that relate to everyday life, and of assessments that include writing and explanation suggests that instruction is meeting the interests and needs of girls.[iv] Third, progress is suggested by indications that girls’ out-of-school science and mathematics experiences were similar in frequency and type to those of boys.[v] Fourth, equal enrollments of boys and girls in high school physics would indicate that the system is becoming more equitable.

Different indicators might be the focus of assessment in an urban system whose identifiable subgroups are African American and white students. Key indicators that such a system is moving toward meeting the needs of the African American girls and boys who are underrepresented in terms of enrollment and achievement in science and mathematics courses are increased enrollments in preschool programs, proportional enrollment and achievement in eighth-grade algebra, availability of science and mathematics courses that meet the national science and mathematics standards, increased representation of African American students in academic programs in high school, a decrease in the acceptance or use of behaviors that detract from learning, and proportional enrollment in calculus.

These two brief examples suggest a sorting of indicators based on identified differences between specific subgroups that are of concern in a given district. The following example describes in more detail how a typical urban system developed and used its equity metric. A pseudonym has been used as confidentiality was promised in working with this district.

Using an Equity Metric: Measuring Central City’s Progress

Central City School Corporation (CCSC) is an urban district that enrolls a mix of students, predominately African Americans (70%) and whites (25%). The district’s elementary, middle, and high schools are divided among magnet schools, neighborhood schools, and neighborhood schools with magnet programs. This complex mix is the result of 20 years of court-ordered desegregation guidelines that imposed quotas on the schools in the district.

When CCSC’s recent tax levy failed, teachers, administrators, and parents met to discuss the future. They agreed that a major goal for the district was high quality science and mathematics education for all students; they also agreed that any reform needed to be systemic, changing the whole system. CCSC began its systemic reform of science and mathematics education by initiating a self-study. The findings indicated extensive tracking of middle and high school students into basic, general, and academic courses in mathematics and science. In addition, data showed that more than half of the African American students failed ninth-grade algebra and biology, compared to 35 percent of white students.

When the state initiated proficiency examinations, higher proportions of African Americans failed them. Further, more than half the students who entered high school dropped out prior to graduation and the rate was higher for African Americans. However, the study also found that the district had a strong program in advanced placement courses and equal numbers of African American and white graduates entered college. (Because data were not disaggregated by race and gender, issues of gender equity had not been identified or addressed.) A potpourri of professional development courses was offered to district teachers by several area universities; however, there was no evidence that courses were screened for effectiveness in improving classroom teaching and/or student learning.

With these data as background, CCSC charted a plan of systemic reform to move toward meeting the needs of all children and equalizing opportunities to learn across courses and schools. Although district administrators and teachers realized that many aspects of the system would need to be evaluated, they chose to begin with two, opportunities to learn and achievement in mathematics and science.

First, a comprehensive assessment plan was created so that baseline data, as well as trend data, were available to chart the progress toward equity in science and mathematics education. Initially, CCSC chose the indicators and measures (shown in Figure 2) to assess academic progress in science and mathematics by race/ethnicity and gender.

   Figure 2: Initial equity plan for Central City School Corporation