Measuring the Success of Small-Group Learning in College-Level SMET Teaching: A Meta-Analysis Leonard Springer National Institute for Science Education Wisconsin Center for Education Research University of Wisconsin-Madison 1025 West Johnson Street Madison, WI 53706 Lspringe@facstaff.wisc.edu Mary Elizabeth Stanne University of Minnesota Samuel Donovan University of Wisconsin-Madison       Student collaboration in college SMET courses and programs is aimed at enhancing the preparation of students for collaboration in SMET professions and at giving all students a better sense of how scientists and engineers work. An American Association for the Advancement of Science report advises that "the collaborative nature of scientific and technological work should be strongly reinforced by frequent group activity in the classroom. Scientists and engineers work mostly in groups and less often as isolated investigators. Similarly, students should gain experience sharing responsibility for learning with each other" (1).       For the most part, college and university educators have yet to respond to calls for greater opportunities for collaboration and cooperation in SMET courses and programs. Regrettably, the unintended consequences of this focus on teaching rather than learning include unfavorable attitudes toward SMET among students, unacceptably high attrition from SMET fields of study (2), inadequate preparation for teaching science and mathematics at the precollege level, and graduates who "go out into the workforce ill-prepared to solve real problems in a cooperative way, lacking the skills and motivation to continue learning" (3).       This meta-analysis of research on college students in SMET is intended to facilitate a greater understanding of the effects of small-group learning at the postsecondary level. We choose to use meta-analysis because the procedure has considerable utility in informing policy and practice (4,5). Conceptual framework       An annotated bibliography of SMET resources in higher education (6) identifies several types of small-group learning practices currently in use. In this meta-analysis, we include cohort groups, various types of structured cooperative learning, brief activities for pairs of students during breaks in lectures, and several types of informal collaborative work among students.       The two sets of research questions guiding the meta-analysis focus on undergraduates in SMET courses and programs. First, we address the main effects of small-group learning on three broad categories of outcomes among SMET undergraduates: achievement, persistence, and attitudes. Second, we address four categories of conditional effects of small-group learning. Procedures       We screened a wide variety of electronic and print resources to identify references for possible inclusion in this study, including ERIC, Education Index, PsycLIT, Dissertation Abstracts International, Medline, CINAHL (nursing and allied health), and ASEE (American Society for Engineering Education) conference proceedings. In addition, we reviewed the reference sections of the myriad studies that we collected in an effort to identify other potentially relevant research. Finally, we contacted several researchers and practitioners who are active in the field and asked them to provide relevant research or to identify additional sources of studies.       Five criteria determined whether a research report qualified for inclusion in the meta-analysis. First, the study examined undergraduates in science, mathematics, engineering, or technology courses or degree programs at accredited postsecondary institutions in North America. Second, studies must have incorporated small-group work inside or outside of the classroom. Third, the study was conducted in an actual classroom or programmatic setting rather than under more controlled laboratory conditions. Fourth, the research was published or reported in 1980 or later on the grounds that recent studies may be more relevant to the current global context in which students learn. Fifth, the research reports enough statistical information to estimate effect sizes.       The metric that we used to estimate and describe the effects of small-group learning was the standardized mean difference (d-index) effect size (7).1       The studies that we collected were coded by an analyst with extensive experience coding and analyzing research on small-group learning. Two additional analysts independently checked the coding that we employed for this study. We resolved occasional differences through consensus. Results       The literature search produced 383 reports related to small-group learning in postsecondary SMET from 1980 or later, 39 (10.2%) of which met the inclusion criteria for this meta-analysis. Of the 39 studies that we analyzed, 37 (94.9%) presented data on achievement, 9 (23.1%) on persistence or retention, and 11 (28.2%) on attitudes. Most of the reports that we retrieved did not qualify for inclusion because they were not based on research.2 Main effect of small-group learning       The main effect of small-group learning on achievement, persistence, and attitudes among undergraduates in SMET was significant and positive (see Figure 1). Based on 49 independent samples, from 37 studies encompassing 116 separate findings, students who learned in small groups demonstrated greater achievement (d = 0.51) than students who were exposed to instruction without cooperative or collaborative grouping. Similarly, based on 12 independent samples, from 11 studies encompassing 40 findings, students in small groups expressed more favorable attitudes (d = 0.55) than their counterparts in other courses or programs. Finally, based on 10 independent samples and findings from 9 studies, students who worked in small groups persisted through SMET courses or programs to a greater extent (d = 0.46) than students who did not work cooperatively or collaboratively. These weighted effect sizes did not differ substantially from the unweighted findings.3 Similarly, redundant effect sizes, based on all nonindependent findings, were comparable to those for the nonredundant or aggregated findings, based on the independent samples reported above.4

Figure 1. Distribution of Effect Sizes.

	Distribution of effect sizes       We tested potential conditional effects of small-group learning using homogeneity analysis (8).5 The results of the homogeneity analysis suggest that the distribution of effect sizes for persistence-related outcomes can reasonably be attributed to chance or sampling error alone. The results also suggest that further grouping of the achievement and attitudinal data is necessary to understand the conditional effects of small-group learning. As indicated by statistically significant Qt statistics, one or more factors other than chance or sampling error account for the heterogeneous distribution of effect sizes for achievement and attitudes.       Methodological factors. Our analyses of the conditional effects of small-group learning suggested that significant variation in effect sizes for achievement-related outcomes can be attributed to method-related influences. Studies that identified the investigator as the instructor reported significantly greater effect sizes (d = 0.73) than studies that did not report the investigator as directly involved in instruction (d = 0.41). Studies that contrasted an experimental and control group (two-sample research designs) reported significantly greater effects (d = 0.57) than studies that analyzed pretests and postests from a single sample (d = 0.30). Investigations undertaken at four-year institutions were associated with significantly greater effects (d = 0.54) than those at two-year colleges (d = 0.21). Importantly, based on data from 276 students representing seven independent samples at six two-year colleges, the average weighted effect size of 0.21 was one of only two statistically nonsignificant results of small-group work reported in our entire study.       Similar to the data on achievement, much of the variance in effect sizes for attitudinal outcomes was associated with methodological factors. Unlike the data on achievement, however, the effects of small-group learning differed significantly among SMET fields of study. The average weighted effect size (d) in the sciences was 0.87, compared with 0.62 in allied health, 0.43 in mathematics, and 0.25 in engineering. The attitudinal data also showed evidence of publication bias, with greater effects reported in journals (d = 0.77) than in other sources (d = 0.42). All attitudinal studies originated at four-year institutions, precluding an analysis by institutional type.       Groups of students. No significant difference in the positive effects of small-group learning on students' achievement was evident between predominantly female (d = 0.39) and heterogeneous or mixed gender groups (d = 0.55). The benefits of small-group learning on students' attitudes, however, were greater for predominantly female groups (d = 0.72) than groups of mixed gender (d = 0.44). This difference is due primarily to the results from a single study.       The positive effect of small-group learning on students' achievement was significantly greater for groups composed primarily or exclusively of African Americans and Latinos/as (d = 0.76) compared with predominantly white (d = 0.46) and relatively heterogeneous (d = 0.42) groups. Sufficient data were not available to analyze the conditional effects of the racial or ethnic composition of groups on students' attitudes.       Small-group learning procedures. There was a higher average weighted effect for supplemental instruction (d = 0.65)--typically study sessions outside of class--than for in-class instruction (d = 0.44). The pattern of differences was reversed for attitudinal outcomes: more favorable effects on attitudes were evident for in-class instruction (d = 0.59) than for supplemental instruction (d = 0.24). The data suggested that greater time spent working in groups had significantly more favorable effects on students' attitudes, with effect sizes of 0.77 for high group time, 0.26 for medium, and 0.37 for low. No significant association between time spent in groups and achievement was evident.       Outcome measures. The effects of small-group learning on achievement were significantly greater when measured with exams or grades (d = 0.59) than with the standardized instruments (d = 0.33). Although small-group work among students had significant and positive effects on students' attitudes toward learning the material (d = 0.56) and their self-esteem (d = 0.61), the effect on their motivation to achieve (d = 0.18) was one of only two nonsignificant results of small-group work that we report in this study. Discussion and Conclusions       The results suggest that small-group learning is effective in undergraduate SMET courses and programs, and support more widespread implementation of small-group learning in undergraduate SMET. Students who learn in small groups generally demonstrate greater academic achievement, express more favorable attitudes toward learning, and persist through SMET courses or programs to a greater extent than their more traditionally taught counterparts. The reported effects are relatively large in research on educational innovation and have a great deal of practical significance. The 0.51 effect of small-group learning on achievement reported in this study would move a student from the 50th percentile to the 70th on a standardized test. Similarly, a 0.46 effect on students' persistence is enough to reduce attrition from SMET courses and programs by 22%. The 0.55 effect on students' attitudes far exceeds the average effect of 0.28 (9) for classroom-based educational interventions on affective outcome measures.       The main effect of small-group learning on achievement is particularly robust, as suggested by analyses of the potential influence of unretrieved studies, commonly known as the file drawer problem. Analyses of the file drawer (10) indicate that 29 independent samples reporting zero-effect-sizes not identified by our search would be needed to lower the average weighted effect size for achievement from 0.51 to 0.32, an effect size that is not considered practically significant. Unretrieved reports of zero-effect-size from four independent samples would be needed to lower the average weighted effect size for persistence from 0.46 to 0.32, and nine would be needed to lower the average weighted effect size for attitudinal outcomes from 0.55 to 0.32. Given the scope of our search for qualified research and the consistently positive effects reported across independent samples, it is unlikely that unretrieved studies would have a substantial impact on the magnitude of the effects that we report.       The results have particularly important implications for policy and practice because they are consistent with the proposition that small-group work is warranted during the first year of college for all students in SMET courses and programs. In addition, the results suggest that small-group learning may have particularly large effects on the academic achievement of members of underrepresented groups and the learning-related attitudes of women and preservice teachers. Moreover, our analysis of small-group learning procedures suggests that greater time spent working in groups leads to more favorable attitudes among students in general and that even minimal group work can have positive effects on student achievement. Furthermore, small-group learning can reduce attrition in SMET courses and programs substantially.       The primary challenge is in moving from analysis to action. The magnitude of the effects reported in this study exceeds most findings in comparable reviews of research on educational innovations and supports more widespread implementation of small-group learning in undergraduate SMET. Small-group learning is clearly successful in a great variety of forms and settings, and holds considerable promise for improving undergraduate SMET education. As recommended by the National Research Council, "Innovations and successes in education need to spread with the speed and efficiency of new research results" (11). Effective action will require bridges among policymakers at national, state, institutional, and departmental levels, and practitioners and scholars across the disciplines. Through collaboration among representatives of these diverse groups, progress can be made toward promoting broader implementation of small-group learning. References 1.   American Assn for the Adv of Science. (1989). Science for All Americans: Project 2061 (Oxford University Press, New York, 1989), p. 148. 2.   E. Seymour and N. Hewitt. Talking About Leaving: Factors Contributing to High Attrition Rates Among Science, Mathematics, and Engineering Undergraduate Majors (Bureau of Sociological Research, University of Colorado, Boulder, 1994). 3.   Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering, and Technology (National Science Foundation, Washington, DC, 1996), p. iii. 4.   C. C. Mann, Science 266, 960-962 (1994). 5.   National Research Council, Combining Information: Statistical Issues and Opportunities for Research (National Academy Press, Washington, DC, 1992). 6.   J. Cooper and P. Robinson, Annotated Bibliography of Science, Mathematics, Engineering, and Technology Resources in Higher Education (California State University, Dominguez Hills, Los Angeles, 1997). 7.   J. Cohen, Statistical Power Analysis for the Behavioral Sciences (Erlbaum, Hillsdale, NJ, ed. 2, 1988). 8.   H. Cooper, Integrating Research: A Guide for Literature Reviews(Sage, Newbury Park, CA, 1989). 9.   J. A. Hattie, H. W. Marsh, J. T. Neill, G. E. Richards, Review of Educational Research 67, 43-87 (1997). 10.   R. G. Orwin, Journal of Educational Statistics 8, 157-159 (1983). 11.   National Research Council, From Analysis to Action: Undergraduate Education in Science, Mathematics, Engineering, and Technology (National Academy Press, Washington, DC, 1996), pp. 5-6. Notes       1We calculated effect sizes with weighted and unweighted procedures. In the unweighted procedure, each effect size estimate was weighted equally in calculating the average effect. In the weighted procedure, greater weight was given to effect sizes associated with larger samples based on the assumption that the larger samples more closely approximate actual effects in the student population of interest.       2Studies dated 1980 or later were excluded as follows: 199 (52.0%) did not involve research (including conceptual papers and classroom resources), 92 (24.0%) did not report sufficient quantitative data to estimate effect sizes (including qualitative investigations), 35 (9.1%) were conducted in psychology laboratories, 12 (3.1%) were conducted outside accredited postsecondary institutions in North America, and 6 (1.6%) compared one or more small-group learning methods with each other.       3In the weighted procedure, the nonredundant effect is weighted by the inverse of its variance. Thus, the sample contributes only one effect size weighted proportionally to its sample size. In an analysis that examined the effects of small-group learning on separate findings, however, this sample contributes one effect estimate to each of the two calculations. Thus, the shifting unit approach retains as much data as possible while holding to a minimum any violation of the assumption that the data points are independent.       4Each finding-level effect size was first coded as if it were an independent event. For estimates of the effects of small-group learning on achievement based on independent samples, correlated effect sizes were averaged and reported as nonredundant. Each independent effect size was multiplied by the inverse of its variance, then the sum of these products was divided by the sum of the inverses.       5A statistically significant Qt suggests the need for further division or grouping of the data. Further grouping may be needed by population (e.g., first-year or other students), methodological factor (e.g., research reported in peer-reviewed journal or other source), small-group learning procedure (e.g., time spent learning in groups), type of outcome (e.g., motivation or self-esteem within attitudes), or a range of other potentially relevant factors. Acknowledgements       The research reported in this paper was supported by a cooperative agreement between the National Science Foundation and the University of Wisconsin-Madison (Cooperative Agreement No. RED-9452971). At UW-Madison, the National Institute for Science Education is housed in the Wisconsin Center for Education Research and is a collaborative effort of the College of Agriculture and Life Sciences, the School of Education, The College of Engineering, and the College of Letters and Science. The collaborative effort is also joined by the National Center for Improving Science Education, Washington, DC. Any opinions, findings, or conclusions are those of the authors and do not necessarily reflect the views of the supporting agencies. The authors wish to thank the following reviewers for their helpful comments on earlier drafts of this manuscript: Andrew Porter, James Cooper, Roger Johnson, Karl Smith, Myles Boylan, Arthur Ellis, Jeffrey Braden, Clifton Conrad, and participants in the National Institute for Science Education's 1997 Review Panel on Cooperative Learning. Click here to view this article as a PDF file (useful for printing). More about the authors
	Note: To view the bibliography as a PDF file (Portable Document Format), the Adobe Acrobat Reader plug-in is required. You can download Acrobat Reader for free by linking to the Adobe Home Page and following the installation instructions that they provide. The program is available for Macintosh, Window, DOS, and Sun UNIX platforms.

	Stories	Doing CL	Resources	More Info	FAQs