ABOUT WCER NEWS Events Cover Stories Research News International Research Press WHAT'S THE RESEARCH ON...? PROJECTS All Active Projects All Completed Projects PUBLICATIONS LECTURE SERIES PEOPLE Staff Directory Project Leaders ERG - EVALUATION RESOURCES GROUP RESOURCES Conference Rooms Equipment GRANT SERVICES GRADUATE TRAINING SERVICE UNITS Director's Office Business Office Technical Services Printing & Mail EMPLOYMENT CONTACT INFO MyWCER WORKSPACE LOGIN

Shihmei Barger

June 2009

In 2007 the Madison Metropolitan School District (MMSD) undertook a three-year teacher professional development project called “Building Informed Citizens.” 

A Teaching American History grant from the U.S. Department of Education enabled the district to offer this professional development project that aims to raise student achievement by increasing teachers’ knowledge and understanding of U.S. history.

This joint effort involves the MMSD,  the University of Wisconsin–Whitewater, the Wisconsin Historical Society, and the Wisconsin Veterans Museum. It’s being evaluated by WCER researcher Shihmei Barger. She aims to provide scientifically rigorous evidence on the extent to which project outcomes meet the project goals and objectives.

Each year, the BIC Project focuses on a core topic in traditional U.S. history. The three annual themes are (a)  roots and ideas of American democracy (b) becoming an American—immigration and migration, and (c) America in the world.  BIC professional development activities include full-day workshops during the school year and annual, week-long summer institutes.

To assess whether intervention teachers’ knowledge and quality of instruction has improved, Barger measures (a) content analysis of lesson plans using a 12-point rubric on standards such as teachers’ historical content knowledge, analytic thinking skills, instructional scaffolding skills, and lesson structure; (b) classroom observation and evaluation based on Authentic Instruction and Assessment, which includes higher order thinking, deep knowledge, substantive conversation, and connections to the world beyond the classroom; (c) American History Teacher Test, a 100-item, multiple choice test constructed from the item bank of the Comprehensive Social Studies Assessment Project of the State Collaborative on Assessment and Student Standards; and (d) a teacher survey based on “Learning about the Teaching American History Program: A Questionnaire for Participants” from Evaluation of the Teaching American History Program.

Barger’s initial analysis of teacher lesson plans identified two areas for improvement: historical content knowledge and analytic thinking. When Barger collects and analyzes the next wave of sample lesson plans from BIC teachers she expects to see a 15% increase on all four of the lesson plan standards. 

Barger’s analysis of classroom observations showed variations in the quality of classroom instruction. Individual teacher scores ranged from 6 to 15 points, with a mean score of 10.3 out of 20 points. Areas in need of improvement included ‘substantive conversation’ and ‘connections to the world beyond the classroom.’ Bargerwill observe the same twenty participant teachers and twenty comparison group teachers during Fall 2009 and Spring 2010. She expects to see a 15% increase in their average scores.

Prior to the BIC project, teachers in the intervention and comparison groups scored equivalently on an American history test  (the U.S. Department of Education’s Comprehensive Social Studies Assessment Project). The two groups of teachers demonstrated the same level of knowledge of American history. When they take the test again in June 2010, Barger expects to see a statistically significant difference in scores between the BIC group and the comparison group.

A pre-BIC survey showed that the participating teachers had less confidence in teaching American history and in their general teaching ability than the comparison teachers. That’s even though the BIC teachers report a higher preference for teaching American history.  When Barger surveys both groups of teachers again in June 2010 she expects to see the intervention teachers demonstrate a 20% increase in confidence in teaching American history and a 20% increase in confidence in general teaching ability.

To assess whether teacher participation in BIC has a positive impact on student achievement, Barger measures (a) student performance on the Wisconsin Knowledge Concepts Examinations–Criterion-Referenced Test, and (b) content analysis of student work (intervention group only).

WKCE scores for the 8th grade students of both intervention and comparison group teachers were collected for assessing the three-year cumulative effects of the BIC Project. Students in the intervention group scored an average of 416.57 on the Social Studies Scale Score, which corresponds to the advanced performance level. Students in the comparison group scored an average of 393.34, which corresponds to the proficient performance level. On the Standards Performance Index History metric, the intervention students averaged 75.59 out of 100 whereas the comparison students averaged 65.36 out of 100, which is 15% lower than the intervention students. Similarly, on the Standards Performance Index Political Science and Citizenship metric, the intervention students averaged 76.0 out of 100 while the comparison students averaged 64.4 out of 100, which is 18% lower than the intervention students. Students of intervention teachers are expected to score higher on the Social Studies proficiency category of the WKCE in 2009 than students of comparison group teachers; specifically, students of intervention teachers are expected to attain at least a 10% increase in 2009 in mean score on Social Studies Scale Score, Standards Performance Index (SPI) History, and SPI Political Science and Citizenship.

Barger’s initial analysis of student work showed that students of BIC teachers performed just above the progressing level on American history achievement (a mean of 5 out of 10 points). This year Barger will again collect and analyze sample student work from the BIC teachers. She and the MMSD project director estimate a 15% increase in the quality of student work.