When you plot Hieggelke's students' post-test results on both the Mechanics Baseline and the Force Concept Inventory along with the results of students taught by other faculty who use the interactive engagement approach to physics, Hieggelke's students show as much if not more gain.
--Alan Van Heuvelen (The Ohio State University Department of Physics)

The JJC faculty are striving to make learning meaningful by concentrating on teaching concepts rather than focusing on repetition of formulas and laws. This philosophy also shapes their summative assessment^a practices. Curt explained, for instance, how many flawed assessments test only the students' ability to regurgitate facts without challenging their true knowledge of the subject. His view, which is widely shared, is that having students who can rattle off Newton's Third Law without really understanding it does not constitute evidence of meaningful learning. Summative assessments should instead say something about a student's deeper understanding of these basic laws of physics. Moreover, faculty who are trying out new learning activities intended to help students achieve meaningful understanding can use the results of these tests to gauge the success of their new approach.

Accordingly, Curt uses a number of tests, some of which are used nationally, that are designed to assess conceptual learning in physics. Below he briefly describes the various exams he gives and the results of those tests.

Curt: Now those tests I mentioned, I give those to all my classes--whether it's the liberal arts conceptual physics course, the algebra-trig based course, the course for allied health students or the physics course for engineering students--because they all have components of what I expect them to learn. I use a set of tests at the beginning of the semester. During the semester they take tests that other people have developed . . . and then the final exam. I've extended my final exam from two hours to four. They schedule exams for two-hour time blocks, so I get two of these blocks and give my students some of those assessment tools in those time slots.

Susan: Do you have any data from the various classes that show, in national comparisons, how well your students are learning? Something about the value-added?

Curt: Yeah, my data show rather strong gains.... The problem, which [large] universities don't face, is that I have particularly small classes. So I have to add all those classes together to get a statistically significant picture.

Because class size at JJC is small, it is difficult to obtain statistically significant information from these assessments on a semester basis. However, Professor Alan Van Heuvelen (The Ohio State University Department of Physics), a national expert in this area, informed us that,

When you plot Hieggelke's students' post-test results on both the Mechanics Baseline and the Force Concept Inventory along with the results of students taught by other faculty who use the interactive engagement approach to physics, Hieggelke's students show as much if not more gain. His students' outcomes compare favorably with those of students taught by Eric Mazur at Harvard University, Paul D'Alessandris at Monroe Community College, and Tom O'Kuma at Lee College.

Below we list the names and acronyms for the summative assessment tests that Curt uses and indicate how, and in which physics courses, he uses them. [Note that these tests are the same as those described in the section on formative assessment, where we presented formative assessment as a learning activity. In Resource D, we provide brief descriptions of these tests and information about how to obtain them. Clearly, Curt uses these tools for both formative and summative assessment purposes.]

1. Maryland Physics Expectations Survey (MPEX); Pre/Post-test in Physics 100, 201, 202
2. Force Concept Inventory (FCI); Pre/Post-test in Physics 100, 201
3. Force and Motion Conceptual Evaluation (FMCE ); Pre/Post-test in Physics 100, 201
4. Testing Understanding of Graphs - Kinematics (TUG-K); Post-test in Physics100, 201
5. Mechanics Baseline Test (MBT); Post-test in Physics 201
6. Vector Evaluation - Tools for Scientific Thinking (TST); Post-test in Physics 201
7. Heat and Temperature Conceptual Assessment (HTCE); Post-test in 201
8. Conceptual Survey of Electricity and Magnetism (CSEM); Pre/Post-test in Phys 202
9. Conceptual Survey of Electricity (CSE); Pre/Post-test in Physics 202
10. Conceptual Survey of Magnetism (CSM); Pre/Post-test in Physics 202
11. Determining and Interpreting Resistive Electric Circuits Test (DIRECT); Post-test in Physics 202
12. Electric Circuit Conceptual Assessment (ECCE); Pre/Post-test in Physics 202

Note that Thornton, Sokoloff, and Laws have developed Items 3, 6, 7, and 12. Much of the data obtained using these tests has been collected to help inform the development of the computer-based MBL lab materials that they have produced. Curt and Bill use these tests because they have adapted many of the Thornton, Sokoloff, and Laws lab materials for their courses.

The Maryland Physics Expectations Survey (MPEX, Item 1) is not a test, but rather a survey that attempts to measure students' attitudes before and after taking a physics class. It asks students to assess the degree to which they agree with statements such as, "Physics is relevant to the real world." Published data for this instrument indicate that students tend to agree with this statement before a standard physics course (usually calculus-based) and tend to disagree with it after having completed the course. This outcome is not what most physics professors seek to achieve. MPEX outcomes for JJC physics students go against this trend.

The Force Concept Inventory (FCI, Item 2), and the Force and Motion Conceptual Evaluation (FMCE, Item 3) tests measure related and somewhat overlapping conceptual areas. The FCI and FMCE deal with kinematics and Newtonian thinking. Questions are multiple-choice and are written in non-technical language, but answers are included among attractive distractors that specifically address common-sense misconceptions about physics. The FCI is widely used, and data on student performance on the FCI are available from scores of courses at various institutions across the nation.

To enable consistent comparison of performance on the FCI of students from diverse institutions (from the most to the least selective), Richard Hake of Indiana University introduced an "average normalized gain" factor (Hake, 1998). As noted in the Introduction, Hake developed this factor, which has come to be known in the physics community as the "Hake factor," while researching the difference between traditional physics classes and what he calls "interactive engagement" classes in terms of students' pre-instruction and pot-instruction performance on the FCI. The significance of the Hake factor is that it adjusts for the fact that percentage improvement is normally easier for those who start with lower pre-test scores than for those who initially score quite high.

At JJC, the FCI results for students in Curt's Engineering Physics course (Physics 201) vary a great deal from semester to semester, in part due to the very small class size. However, averaged over six semesters (Fall 1997 - Spring 2000), their Hake factor is 0.47 for the FCI (Table 1, below), which is comparable to the average Hake gain nationally for interactive engagement courses. The Hake factor for the same students on the FMCE, to which Curt has added several questions dealing with momentum is 0.62.

Table 2 provides data on how Engineering Physics students performed on three post-tests:

• Testing Understanding of Graphs - Kinematics (TUG-K, Item 4), which considers kinematics only.

• Mechanics Baseline Test (MBT, Item 5), which claims to measure problem solving and was developed by the authors of the FCI to measure problem solving in Newtonian mechanics.

• Heat and Temperature Conceptual Assessment (HTCE, Item 7), which measures special aspects of physics courses (not widely used, and little published data are available).

Table 3 provides data on how Engineering Physics students performed on three pre/post-tests:

• Conceptual Survey of Electricity and Magnetism (CSEM, Item 8)

• Conceptual Survey of Electricity (CSE, Item 9), and

• Conceptual Survey of Magnetism (CSM, Item 10).

Finally, data for JJC Engineering Physics students on two more tests are shown in Table 4:

• Electric Circuit Conceptual Assessment (ECCE, Item 11), and

• Determining and Interpreting Resistive Electric Circuits Test (DIRECT, Item 12).

These tests measure student learning in the area of circuits and focus on basic concepts and difficulties that students typically have.

Bill's students show gains for algebra/trig-based College Physics course that are comparable to these presented here for Curt's Engineering Physics students. These gains provide evidence that Curt and Bill are achieving their goals for student learning.

(For evidence of positive gains in student attitudes, see The labs are incredible, absolutely incredible: Students Discuss Computer-Dependent Learning Activities.

(For more in-depth discussions of the assessments tools used by Curt, see We have to know where students' problems are and not where we think they are: Curt Discusses Formative Assessment Activities and Once you do the task, you learn it: Students Discuss Formative Assessment Activities.