Vol. 2, No. 3. - August 1999

Measuring Progress Toward Equity in Science and Mathematics Education

By Jane Butler Kahle

October 1997 marked the 40th anniversary of Sputnik, the rocket that provided an early impetus for reform of science and mathematics education in the United States. Forty years and several permutations later, we are still involved in reform. Now the focus is on making reform systemic and enabling all students to gain literacy in mathematics, technology, and science, rather than educating relatively few to become mathematicians, engineers, and scientists.

The reform of the 1960s did not address the interests or needs of many students who, by nature of their culture, gender, or physical or economic condition, were less attuned to, or had less access to, quality science and mathematics education. Rather, classes were tracked and only a few students benefited. In the ensuing 40 years, the numbers of those historically excluded students increased dramatically.

The driving force behind the current reform movement is the need to remain economically, scientifically, and technologically competitive with other developed nations. Increasingly, as K–12 students have become more diverse and as the underrepresentation of whole groups of students in science and mathematics has become more visible, we have come to understand that this time the reform of science and mathematics education must be both systemic and equitable. That is, the reform must address multiple parts of an educational system, and it must increase the access, retention, and achievement of students from all subgroups in high quality science and mathematics programs. Curricula must change to represent varied interests, to implement more effective ways of organizing classrooms and schools and of providing instruction, and to use assessments that include multiple ways of demonstrating learning and competencies. In addition, policies that determine both the quantity of courses and the quality of the educational experience (e.g., teacher qualifications, teaching resources, and academic tracking) must be reviewed and changed to ensure equitable reform. As our student population becomes ever more diverse, simple and defensible ways to measure progress toward meeting the needs and expectations of all students have become increasingly important. Equity, or high quality science and mathematics education for all students, matters in today’s reform.

One way to approach these issues is to take stock and assess where a system is along a continuum toward equity in reform. Each system, defined as a school district in this discussion (but, conceptually, a system may be any educational unit—from an individual class to an entire state), needs to identify guideposts along the path to high quality education in science and mathematics for all students. Taken together, those guideposts form an equity metric, a way to measure progress toward equity.

This Brief proposes and describes a methodology for developing and using equity metrics in ways that measure genuine progress toward high quality science and mathematics education for all students.

Developing an Equity Metric: From Guideposts to Indicators

Guideposts for equity may be found in the analysis of large national and international databases, in research literature, and in the changing policies and practices of the current reforms of science and mathematics education. For purposes of monitoring a system’s progress toward equity, it is important to provide easily understood and acceptable data. Therefore, only measurable guideposts, commonly called indicators, are included in this discussion of equity metrics. These indicators are drawn from three large databases (NELS:88, High School and Beyond, and TIMSS),[i] NSF’s indicators of quality mathematics and science education (National Science Foundation, 1996), and the research literature for evidence of inequality in access, retention, and/or achievement across student subgroups. If evidence of inequity on a type of indicator was found in two or more sources (e.g., unequal enrollments by subgroups in eighth-grade algebra), the indicator has been included in the metric.

Next, the identified indicators have been sorted by grade levels. This helps address two questions:

·          At which grade levels are information about students collected?

·          At which levels are enrollment, participation, and achievement critical for a student’s continued access to and/or progress in science and mathematics?

The sorting suggests leverage points in the educational system that are related to critical times in a child’s education; that is, periods when educational systems routinely gather data concerning specific placement (e.g., general mathematics or algebra) and performance (e.g., standard achievement tests, high school graduation). The leverage points identified here are preschool and  fourth, eighth, tenth, and twelfth grades. Indicators have been sorted by appropriate leverage points.

Lastly, indicators of general reform were identified. Using the above criteria and databases, indicators of systemwide progress have been added to the metric, shown in Figure 1.

  A primary reason for caution when using this approach is that gender differences may not be identified. Girls and boys enroll in equal numbers in algebra, biology, calculus, chemistry, and trigonometry. Further, on average, girls achieve higher grades in those courses than boys do. However, the enrollment patterns in physics are not equal, suggesting that neither course enrollment patterns nor achievement levels in science and mathematics predict girls’ enrollment in physics.[ii]

  Once indicators have been identified, an educational system can select among them to design its own equity metric. The indicators included in the model equity metric in Figure 1 have been selected to meet the following criteria:

·    They are sensitive to diversity among subgroups of students, teachers, and others.

·    They can be used to inform action, not just to define the present state.

·    They are flexible, because not all metrics are relevant to all parts of the system.

·    They distinguish among access, retention, and achievement.

·    They are directed toward leverage points in the system.

·    They are feasible to use (i.e., affordable).

Constructing a Metric: Selecting Indicators

Indicators may vary across time, changing to address different factors and/or conditions. For example, early studies suggested that  teacher qualifications were an indicator of inequity, as they differed between schools serving primarily minority students and those enrolling primarily majority students. However, analysis of current databases indicates that teachers of minority students are not necessarily less well prepared than teachers of majority students in terms of certification, number of years in teaching, or educational level. There are no significant differences on these indicators in science, and the only difference in mathematics is in the percentage of certified teachers of Native American students compared to all other groups. Therefore, instead of using certification, experience, and attainment of a bachelor’s degree as indicators of inequity in teacher qualifications, indicators of the quality of the teacher preparation and professionalization programs may be needed. For example, more useful indicators may include number of credits in science and mathematics courses, evidence of advanced as well as introductory science and mathematics courses in the undergraduate program, length and quality of practicum or intern experience, and certification by the National Science Teachers Association or the National Board for Professional Teaching Standards.

Other indicators, such as Home Resources, may be composed of several factors. For example, attendance at preschool has been found to be an indicator of inequity for Hispanic and Native American children, while presence of a table or desk for a student’s own use and presence of a computer in the home differ between minority and majority students and have been linked to student achievement in many of the 41 countries (including the United States) in the Third International Mathematics and Science Study (TIMSS) (Beaton, Martin et al., 1996; Beaton, Mullis et al., 1996). Those components are easy to measure and may be assessed as part of the indicator.

The indicator Student Attitudes and Beliefs addresses the documented decline in positive attitudes in science between fourth and twelfth grades. It is relatively easy to measure and also can be used to address gender equity, because the decline in attitudes is greater for girls than for boys. [iii]

Another indicator, Learning Behavior, includes absenteeism and tardiness (which are easy to measure and indicate degree of student engagement in learning), the priority students place on learning, and the amount of competition students face for grades (increasing competition correlates with decreasing achievement among non-Asian minority groups).

One of the most interesting indicators is Quantity/Quality of Math/Science Courses. Recent studies suggest that to provide equitable education we must move beyond counting the hours or numbers of courses and assuming that courses with similar titles are comparable. Observational studies, teacher logs, teacher and student surveys, and student portfolios are some of the ways by which we can assess the quality of a course. Although indicators of quality (depth of coverage and mode of instruction) are needed, enrollment in key gatekeeping courses (such as eighth-grade algebra or high school geometry) and the Availability of Advanced Math/Science Courses are also critical indicators of high quality mathematics and science education. Other key indicators, found in Figure 1, are both the intent to enroll in an Academic Program in the eighth grade and actual enrollment in one in the tenth grade.

Quality of Professional Development is included as an overall indicator of movement toward equity. Teachers need access to life-long learning and skill development to implement challenging curriculum, to use varied instructional strategies, to include multiple types of authentic assessments in their classrooms, and to improve their understanding of the backgrounds of students from diverse subgroups. Measurement of the quality of teacher professional development needs to move beyond the number of college or continuing education credits accrued toward the quality of outcomes. Evidence of changing practices, behaviors, and attitudes among teachers and students that may be collected through teacher logs, student journals, audio and video tapes, and interviews is needed. Further, a critical indicator of the quality of professional development is improvement in the retention and achievement of students in all subgroups.

Different Challenges, Different Indicators

Once a system has articulated its equity goals and has identified guideposts or indicators of equity, it must formulate a working plan for becoming more equitable, as well as a timeline for initiating components in its plan. It is estimated that systems will need at least five years to demonstrate progress toward equity, using the indicators in Figure 1. Initially, baseline data and appropriate benchmarks of progress must be identified. Next, ways of monitoring progress are needed. Finally, collection and analysis of data, coupled with dissemination and discussion of the findings, must occur. Fortunately, national databases suggest key indicators as well as ones that are applicable for specific student subgroups.

What are key indicators that any system is becoming more equitable? First, retention and achievement in eighth-grade algebra are key indicators of a student’s probability of achieving a high quality education in mathematics and science. Second, although not easily quantified, the quality of the content of science and mathematics courses is critical. Third, a clear indication of progress is provided by data from achievement tests that show narrowing of gaps concomitant with increased achievement by all subgroups of students. Fourth, evidence that teaching practices are changing in ways that involve students actively in learning is important, because active engagement enhances both interest and achievement levels of students who historically have been underrepresented in science and mathematics (Stevens, 1996). Although it is tempting to continue to identify key indicators, these four will indicate movement toward equity and provide salient guideposts along the way.

Another approach is to look for indicators that address a given system’s priorities. In a rural school system where children have similar ethnic/racial backgrounds and speak English at home, movement toward equity may involve removing differences between girls and boys. What are key indicators of gender equity? First, given that girls exhibit a greater decline than boys in attitudes about, and interest in, science, a key indicator of gender equity is sustained positive attitudes and interest levels as girls proceed from fourth grade (where girls are as positive about science and as interested as boys are) through high school. Second, evidence of cooperative learning groups, of activities that relate to everyday life, and of assessments that include writing and explanation suggests that instruction is meeting the interests and needs of girls.[iv] Third, progress is suggested by indications that girls’ out-of-school science and mathematics experiences were similar in frequency and type to those of boys.[v] Fourth, equal enrollments of boys and girls in high school physics would indicate that the system is becoming more equitable.

Different indicators might be the focus of assessment in an urban system whose identifiable subgroups are African American and white students. Key indicators that such a system is moving toward meeting the needs of the African American girls and boys who are underrepresented in terms of enrollment and achievement in science and mathematics courses are increased enrollments in preschool programs, proportional enrollment and achievement in eighth-grade algebra, availability of science and mathematics courses that meet the national science and mathematics standards, increased representation of African American students in academic programs in high school, a decrease in the acceptance or use of behaviors that detract from learning, and proportional enrollment in calculus.

These two brief examples suggest a sorting of indicators based on identified differences between specific subgroups that are of concern in a given district. The following example describes in more detail how a typical urban system developed and used its equity metric. A pseudonym has been used as confidentiality was promised in working with this district.

Using an Equity Metric: Measuring Central City’s Progress

Central City School Corporation (CCSC) is an urban district that enrolls a mix of students, predominately African Americans (70%) and whites (25%). The district’s elementary, middle, and high schools are divided among magnet schools, neighborhood schools, and neighborhood schools with magnet programs. This complex mix is the result of 20 years of court-ordered desegregation guidelines that imposed quotas on the schools in the district.

When CCSC’s recent tax levy failed, teachers, administrators, and parents met to discuss the future. They agreed that a major goal for the district was high quality science and mathematics education for all students; they also agreed that any reform needed to be systemic, changing the whole system. CCSC began its systemic reform of science and mathematics education by initiating a self-study. The findings indicated extensive tracking of middle and high school students into basic, general, and academic courses in mathematics and science. In addition, data showed that more than half of the African American students failed ninth-grade algebra and biology, compared to 35 percent of white students.

When the state initiated proficiency examinations, higher proportions of African Americans failed them. Further, more than half the students who entered high school dropped out prior to graduation and the rate was higher for African Americans. However, the study also found that the district had a strong program in advanced placement courses and equal numbers of African American and white graduates entered college. (Because data were not disaggregated by race and gender, issues of gender equity had not been identified or addressed.) A potpourri of professional development courses was offered to district teachers by several area universities; however, there was no evidence that courses were screened for effectiveness in improving classroom teaching and/or student learning.

With these data as background, CCSC charted a plan of systemic reform to move toward meeting the needs of all children and equalizing opportunities to learn across courses and schools. Although district administrators and teachers realized that many aspects of the system would need to be evaluated, they chose to begin with two, opportunities to learn and achievement in mathematics and science.

First, a comprehensive assessment plan was created so that baseline data, as well as trend data, were available to chart the progress toward equity in science and mathematics education. Initially, CCSC chose the indicators and measures (shown in Figure 2) to assess academic progress in science and mathematics by race/ethnicity and gender.

   Figure 2: Initial equity plan for Central City School Corporation

As data were collected, they were analyzed by both race and sex to identify any differences among subgroups, and individual school data were returned to the principals and teachers for discussion and action. As the reform progressed, CCSC (with its union’s support) requested that schools set individual equity goals and provided incentives for reaching them. Principals’ raises were linked to improvement, as were school-based bonuses. (Union-negotiated contracts prohibited individual teacher bonuses.)

CCSC instituted curriculum reforms (both content and instruction) and developed mechanisms for monitoring progress. All remedial and general mathematics and science courses were replaced with academic courses, and reviews of student transcripts provided progress data. Research-validated, inquiry-based curricula were identified and professional development was provided for school-based teams of teachers.[vi] Teachers kept logs of their teaching activities and strategies, and the district surveyed a random sample of teacher logs and student portfolios to assess changes in teaching practice and in the implemented curriculum.

To address the critical issue of unacceptably high failure rates in biology and algebra, as well as high school dropout rates, the district collected data on student mobility and began to allow students to complete the school year in the same school, regardless of geographic boundaries. Elementary and middle schools were reorganized into multilevel teams so that teachers and students had the opportunity to become learning communities, providing stability and a nurturing environment that was effective in lowering both absentee and dropout rates. Attitudinal data (interest in science, confidence in science skills, perceptions of scientists), behavioral data (numbers of in- or out-of-school suspensions), and attendance data (by specific course) were collected to indicate progress or problems by subgroups. Further, the school system instituted summer programs for eighth-grade students who were at risk of failing ninth-grade algebra and/or biology. The failure rates dropped precipitously, indicating movement toward equity and the need for similar bridge programs throughout high school.[vii]

As the reform matured, analyses of teaching practice and achievement data continued to identify leverage points in the system. In addition, it was possible to compare the positive effect of a critical mass of minority students in a calculus class on their achievement and future educational goals and to change boundaries and scheduling to ensure a critical mass in other indicator courses.

As the district’s white population became increasingly Appalachian, appropriate indicators were added to the equity plan. For example, attendance in preschool, students’ beliefs about the usefulness of mathematics and science, and course selection patterns were monitored for indications of inequity in that emerging subgroup of students.

Early in its reform, CCSC found that past measures of student achievement did not reflect the content of its new inquiry-based curricula. CCSC valued student achievement at the fourth, eighth, tenth, and twelfth grades as indicators of progress and problems, but it needed new achievement measures such as tests composed of public-release National Assessment of Educational Progress (NAEP) or TIMSS items or new performance-based assessments.

Three to four years into its reform, CCSC’s equity metric indicated progress by student subgroups in meeting high standards in mathematics and science. The metric evolved as CCSC’s reform evolved, providing useful guidelines and practical measures of progress toward equity. Further, by setting high goals and standards, by systematically measuring progress, and by addressing the needs of emerging subgroups, CCSC garnered community support for its systemic reform.

Summing It Up: Why Is Equity Important in Systemic Reform?

In biology, systemic means affecting all systems (nervous, digestive, etc.), and each system has self-correcting feedback mechanisms. In education, systemic reform also refers to the whole system, affecting all parts. An equity metric may be used by administrators and teachers to provide continuous feedback during systemic reform, informing and changing components as needed, addressing and correcting inequities, and evolving and adapting indicators and measures. It is not the one, nor only, solution, but it may allow reformers to assess progress and to alleviate problems in providing equitable education in science and mathematics for all students.
For Further Reading

Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Smith, T. A., & Kelly, D. L. (1996). Science achievement in the middle school years: IEA’s Third International Mathematics and Science Study. Chestnut Hill, MA: Boston College, TIMSS International Study Center.

Beaton, A. E., Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1996). Mathematics achievement in the middle school years: IEA’s Third International Mathematics and Science Study. Chestnut Hill, MA: Boston College, TIMSS International Study Center.

Fennema, E. (1990). Justice, equity, and mathematics education. In E. Fennema & G. C. Leder (Eds.), Mathematics and gender (pp. 1-9). New York: Teachers College Press.

Ingels, S. J., Abraham, S. Y., Karr, R., Spencer, B. D., & Frankel, M. R. (1989). National Education Longitudinal Study of 1988: Data file user’s manual. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics.

Johnson, R. S. (1996). Setting our sights: Measuring equity in school change. Los Angeles, CA: The Achievement Council.

Kahle, J. B. (1996). Opportunities and obstacles: Science education in the schools. In C. S. Davis, A. Ginorio, C. Hollenshead, B. Lazarus, & P. Rayman (Eds.), The equity equation: Fostering the advancement of women in the sciences, mathematics, and engineering (pp. 57-95). San Francisco: Jossey-Bass.

Kahle, J. B. (1998). Equitable systemic reform in science and mathematics: Assessing progress. Journal of Women and Minorities in Science and Engineering, 4(2-3), 91-112.

National Science Foundation (NSF). (1996). REC Indicator Series: The learning curve: What we are discovering about U.S. science and mathematics education. Washington, DC: Author.

Oakes, J., Ormseth, R., & Campbell, P. (1990). Multiplying inequalities: The effects of race, social class, and tracking on opportunities to learn mathematics and science. Santa Monica, CA: The RAND Corporation.

Peng, S. S., Fetters, W. B., & Kolstad, A. J. (1981). High school & beyond: A national longitudinal study for the 1980s. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Schmidt, W. H., McKnight, C. C., & Raizen, S. A. (1997). A splintered vision: An investigation of U.S. science and mathematics education. Boston: Kluwer Academic Publishers.

Stevens, F. I. (1996). Closing the achievement gap: Opportunity to learn, standards, and assessment. In B. Williams (Ed.), Closing the achievement gap: A vision for changing beliefs and practices (pp. 77-95). Alexandria, VA: Association for Supervision and Curriculum Development.

Wellesley College Center for Research on Women. (1992). How schools shortchange girls. Washington, DC: American Association of University Women Educational Foundation.

NISE Brief Staff

This Brief was supported by a cooperative agreement between the National Science Foundation and the University of Wisconsin-Madison (Cooperative Agreement No. RED-9452971). At UW-Madison, the National Institute for Science Education is housed in the Wisconsin Center for Education Research and is a collaborative effort of the College of Agricultural and Life Sciences, the School of Education, the College of Engineering, and the College of Letters and Science. The collaborative effort also is joined by the National Center for Improving Science Education in Washington, DC. Any opinions, findings or conclusions herein are those of the author(s) and do not necessarily reflect the views of the supporting agencies.

No copyright is claimed on the contents of the NISE Brief. In reproducing articles, please use the following credit: "Reprinted with permission from the NISE Brief, published by the National Institute for Science Education, UW–Madison." If you reprint, please send a copy of the reprint to the NISE.

This publication is free on request.

National Institute for Science Education
University of Wisconsin-Madison
1025 W. Johnson Street
Madison, WI 53706
(608) 263-9250
(608) 263-1028
FAX: (608) 262-7428

Email: niseinfo@macc.wisc.edu