By September 1999, the new curriculum had become the required program for approximately 80% of first-year engineering majors at UMD. Expansion produced some unexpected challenges and the paper will show assessment data indicating both positive and negative changes in performance in various aspects of the program. We will give insight into the problems and opportunities that developed as the program grew. We will also describe how assessment provided feedback to help decision making.

I. Introduction

After several years of development, the University of Massachusetts Dartmouth (UMD) began a successful, integrated, first year engineering curriculum in September 1998. This new program was called IMPULSE (Integrated Math, Physics and Undergraduate Laboratory Science, and Engineering). The new curriculum dramatically changed the freshman year because it included

• integrating multiple subjects

• teaching and using teamwork among students and faculty

• using technology-assisted classrooms to accelerate learning

• using active and cooperative learning^a

• encouraging formation of a learning community of students and faculty

• using rigorous assessment to evaluate and improve performance.

Forty-eight calculus-ready engineering students began the pilot curriculum in September 1998 and by midterm it was obvious that the program was having a remarkable effect. Only one student had dropped any course and most of the time all remaining students were in every class every day. Details of assessment data will be discussed later in this paper but, after the first semester, results were so positive that a modified IMPULSE program was made the required curriculum for all electrical, computer and mechanical engineering and physics majors.

As a result, IMPULSE was expanded the following year. Eighty-seven calculus-ready students started IMPULSE in September 1999. Then forty-one students started it in January because they had taken precalculus in the fall and could not enter earlier. The same pattern was repeated the third year but with approximately 10% more students.

II. The IMPULSE Pilot Versus the Traditional Program

Table I shows the basic structure of the traditional program for most engineering majors. Each major had its introductory course in the first semester and specified additional unique courses during the first year. Classes typically involved large amounts of straight lectures and, depending on the particular instructors, various amounts of hands-on activity and the use of technology in the classroom.

The pilot IMPULSE first-year curriculum is shown in Table II. We will summarize some relevant points here. Additional details about the new courses and their innovations can be found in previous ^b papers on ^c IMPULSE^d.

A fundamental difference between IMPULSE and the traditional program was that it integrated and sequenced nearly all courses carefully together. The integrated courses are shown in Italics in the table. Physics was used to motivate and enhance students' intuition for calculus and to allow a calculus-based physics to be taken at the same time as calculus. An engineering course in each semester was also integrated to motivate learning of science and math fundamentals while providing engineering foundations. Engineering problems were developed that required knowledge and methods from the other courses. For example, calculus was sequenced to provide "just-in-time" development of the mathematics to deal with physics and engineering problems. In addition, papers were required in the technical subjects and these were worked on and graded jointly in the English course.

In order to keep students' loads reasonable, the first chemistry course was revised to reduce the number of hours students spent in class. IMPULSE chemistry met three hours per week, had two wet lab experiences and used computer tools extensively for exercises, activities and visualization. Traditional chemistry had the usual lecture classes, recitations and laboratories totaling seven hours per week.

Students in the pilot could not drop any IMPULSE course except chemistry because of the integration of subjects. Chemistry was more loosely integrated so that most of its content was not necessary for the other courses.

Another important element of IMPULSE was that students were put in teams of three to four and these teams were used across all courses in a single semester. They were taught about how to work in teams and to be responsible for their own education and that of their teammates. In order to encourage teamwork and group problem solving outside of class, those IMPULSE students who lived on campus were also co-located in the dorm.

Each subject was taught by a faculty member from that discipline and separate grades were issued in each course. Nonetheless, some assignments were made and worked on in more than one course. Those results were sometimes included in grading for all courses involved.

Integration of subjects required faculty effort during development of the syllabus and throughout the semester. During the pilot, IMPULSE instructors met once per week to coordinate the integration of their subjects. In this way, they could point to material from the other courses and expect students to use it. They could also set up problems so that another course could provide "just-in-time" learning. Weekly meetings were also used to coordinate strategies to resolve student and team performance problems.

The engineering course was the motivating force in the curriculum, and in that sense it was critical to student's perception and activity. In addition, the engineering instructor provided leadership to the program and called meetings and coordinated schedules between subjects as needed. This instructor also served as advisor and counselor to all of the students in the course, taking the lead to ensure that student or team performance issues got attention. Sometimes these issues required coordinated activity with the other instructors. Sometimes they could be done directly.

The Engineering Courses in the Pilot:
The first engineering course emphasized problem-solving skills in engineering mechanics in the fall semester. Students met every other day for a total of five hours a week. There were two-contiguous class hours on the first and second day, and one hour on the third day. Occasional special events were also arranged outside class times.

The two-hour classes were used for development of skills in analysis, solid modeling, or design and for integrated problems with Newtonian physics and calculus. The one-hour class was designated for special events such as team building activities or presentations by professionals from industry. Professionals presented on topics such as what engineers do on a day-to-day basis, what research engineers do, how engineers deal with patents and invention, women in engineering, and ethics in engineering. The students showed a profound interest in these topics. During the first pilot semester, eight engineers were invited.

During the semester there were impromptu engineering design projects. That is, students were not notified in advance about the nature of a design problem but were challenged to come up with a design spontaneously in a limited time period. For example, in one early project in a special extended class period, students were told to bring swimsuits but were not told why. In class they were given a limited amount of materials and were asked to design a cardboard canoe. They had two hours to design and build it. That was followed by a competition in the university's swimming pool. All in all, this brainstorm-design-build-compete activity took five hours. The students developed a feel for design, what can fail, and the consequences of failure.

Afterward, the students were asked to work as a team on an improved version of their first design and get ready for a second competition in two weeks. Aside from camaraderie and team building, the students learned first-hand about the importance of understanding and using relevant concepts in their design.

In the second course offered in the spring semester, students met for four hours a week. The course had a mechatronics theme and emphasized development of problem solving skills in AC and DC circuits, electromagnetism, software tools, measurement and controls. The course was structured with two-hour classes on two alternating days. The course involved considerable design with hands-on activity in class but it did not have the extra activities like those that occurred in the first semester.

III. The Expanded IMPULSE Program

Table III shows the modified version of the IMPULSE program that was adopted as the required first year for all students majoring in mechanical, electrical and computer engineering and physics. Three courses remained tightly integrated, physics, calculus and engineering, as shown in Italics. Integration of the English course was dropped because some entering calculus-ready students do not pass the placement tests and are assigned to remedial English. That would have prevented some technically competent students from going into IMPULSE in the first semester. Nonetheless, as the program continues to grow that decision will likely be revisited. Integration of significant writing and oral presentation experiences across the technical courses, with support from an English instructor, provided powerful motivation for students to learn to communicate well.

Chemistry was never integrated fully and its independence was formalized in the expanded program. The first course, however, used cooperative learning and contained other innovations as described above. It was kept the same as the pilot except that its students were not necessarily also in IMPULSE and it developed its own teams. The second IMPULSE chemistry course was changed back to the traditional one because it had a similar basic structure with its associated formal wet lab. It did not appear to be necessary to maintain a separate course in that case.

The traditional courses in physics and calculus continue to be available in parallel with IMPULSE each semester. They are needed by science and math majors. Therefore transfer students and others who already have taken one or more of the three integrated IMPULSE courses have been placed in the traditional courses as necessary. This is also true for students who failed only one or two of the integrated courses in IMPULSE.

IV. The Impact of Rapid Expansion

The IMPULSE program was adopted in late spring of 1999 and it was decided to implement it at full scale in the fall semester of the same year. Such rapid expansion generated considerable stress on faculty and administration to get everything in place for fall classes. Another technology-oriented classroom was constructed so that the engineering, physics, and calculus courses could run for two cohorts for a maximum of ninety-six students. A trailing section in the spring semester would also add another cohort of up to forty-eight. As a result, the number of faculty members involved in the program more than doubled and their other pre-existing teaching schedules forced some creative scheduling of IMPULSE courses.

Since students were scheduled as a cohort for three courses and these were widely separated in time, it became very difficult to do many of the special events. For example, carrying out the cardboard canoe design project became impossible because of scheduling problems.

During the first semester, a common hour was not available for the two different sections of IMPULSE. Fewer professionals could be arranged to speak to each cohort because of the difficulty of scheduling them for both sections.

Most of these problems extended into the second semester of the second year of IMPULSE and into the third year as the program continued to expand for additional cohorts. To resolve some of these issues, two more technology classrooms have been built to handle IMPULSE-style courses, making a total of four in the College of Engineering.

When the pilot of IMPULSE began, all of the instructors involved had worked hard to construct an effective program and they knew a great deal about the methods they were going to use. For example, in preparation they had attended several workshops on active-cooperative learning and using teaming in the classroom. These faculty members met weekly to discuss issues related to management of the program, integration, teaming efficacy, frequency of lecturing, impact of technology on pedagogy, and how to cope with dysfunctional teams.

As the program expanded and more instructors became involved, some with little warning, training and coordination became issues of importance. The added faculty often had little training or experience with active-cooperative learning and teaming. The approach used to compensate was to have the experienced professors mentor the new ones in their subject as the semester got underway. This had varied results depending on the personalities of the people involved. The most effective approach, however, was used by a physics professor who was retiring after the first year. He arranged for his replacement to attend class and assist two or three times during the semester before his retirement. The new instructor experienced the reality of cooperative learning in a team-based class. This was the most effective training model.

In the second year, the integrated subjects were not coordinated together as tightly as during the pilot. Faculty members met less often with those teaching the other integrated subjects for their cohort. In addition, the newest faculty to IMPULSE, and the most inexperienced in cooperative learning, met the least often and were less able to handle student problems and dysfunctional teams.

V. Assessment Results

From the beginning, considerable effort has been put into assessing the IMPULSE program so that good decisions could be made about its expansion or modification. Measurements are being made on an ongoing basis using a variety of devices.

The data summarized here includes results from the first two years of IMPULSE, including the pilot. Previous papers^e should be consulted for more details on the pilot program assessment. The information will continue to be updated and included in the conference presentation.

After a study of the factors that correlated with academic performance of first-time-full-time freshman engineering majors from 1997-98, we developed comparison groups. These groups were matched for their calculus placement entrance test score (CP) and high school GPA as follows:

• IMPULSE I - 48 calculus-ready engineering majors in the pilot, CP=70.4%, H.S.GPA=3.03

• F'98 control - 42 science, math and engineering majors in traditional courses, CP=69.2%, H.S.GPA=3.01

• F'97 control - 38 engineering majors in traditional courses, CP=69.2%, H.S.GPA=2.99.

They also matched closely in SAT math and verbal scores. The F'97 control would have been IMPULSE students if the program had started a year earlier.

The F'98 control group contained very few engineering majors. We have made several studies of calculus data from 1997 and 1998 and the analysis supports the use of science majors for comparison groups to assess the performance of engineering students. None of these studies indicates that engineering freshmen perform differently than science or math majors in calculus.

IMPULSE students were randomly selected from the calculus-ready population of first-time-full-time engineering majors. All of those selected started the program.

For comparison the expanded IMPULSE II in 1999-2000 had the following scores:

• IMPULSE II - 87 calculus-ready engineering majors, CP=64.4%, H.S.GPA=3.16

Calculus:
As shown in figure 1, IMPULSE I and II students scored almost a grade and a half higher than the F'98 control group on 18 common exam questions on the final exam for all sections of the first calculus course. Only 4% of IMPULSE students in each year did not take this final compared to 28% of the F'98 control.

Figure 1

Physics:
Fair comparison with traditional physics courses is difficult. IMPULSE students are the only students who were taking physics during the first semester of their freshman year. Comparison is further complicated because the IMPULSE development caused changes in the way traditional physics classes were being taught. Active learning techniques were first introduced in the spring of 1998 and exercises similar to those in

Figure 2

IMPULSE physics were introduced in standard physics courses in the fall of 1998. For these reasons we chose the comparison group:

• S'97 physics class - 74 students who took PHY 113 that semester (72% were engineering majors, 82% were freshmen)

The Force Concept Inventory (FCI) was used for comparison of learning in first semester physics courses through 1998 and IMPULSE I. This test uses conceptual questions to determine the depth of understanding of Newtonian mechanics. It was given as a pre-test at the beginning of the first semester of physics and again as a post-test at the end. A normalized gain was computed by taking the gain from pre- to post-test and dividing by the maximum possible gain on the post-test (perfect score minus pre-test score).

As shown in figure 2, IMPULSE I students had a normalized gain on the FCI of 30% for the pre-test/post-test pair. In comparison, the S'97 physics class, using traditional methods, had an 18% normalized gain.

Beginning with IMPULSE II, assessment of first semester physics was done with the Force and Motion Conceptual Evaluation^f (FMCE) which in many ways is similar to the FCI. Physics faculty members at UMD believed it was more comprehensive than the FCI because it had a larger number of questions and they were more directly related to the laboratory modules used in first semester physics. IMPULSE II students scored a 32.5% normalized gain on the FMCE.

In evaluating the results above, it should be noted that 98% of the students in both years of IMPULSE took the first semester physics final. While the number of students actually taking the final was not recorded for the comparison S'97 class, traditional physics classes typically have from 65% to 85% take the final.

Engineering:
As discussed above, the IMPULSE courses were so different from the department specific courses that no direct comparison of course results was attempted. Assessment of these courses is currently being developed and is directed toward continuous improvement of the program.

Student Success Rates in the First Semester:
As shown in figure 3, students in IMPULSE have consistently earned substantially more credits during the first semester than the control groups. Students in IMPULSE I and II were also more successful in passing the entire first-year calculus and physics sequences on time, as shown in figure 4. Nonetheless, there was a drop in the number of IMPULSE students passing the physics sequence on schedule during the second year of the program.

Figure 3

Figure 4

The larger number of credits earned by IMPULSE students, and their higher success in passing both physics and calculus courses on time, are even more significant given the improved performance on common exams noted above. In addition, IMPULSE students were taking three very difficult courses at the same time - physics, chemistry and calculus - while the control groups took chemistry and calculus but not physics. In the traditional programs, engineering majors typically take physics in their second semester and most science majors take it in their third semester.

Retention:
As shown in figure 5, the retention of IMPULSE students has been substantially improved over those going through the traditional program; however, the one-year retention for the IMPULSE pilot was much greater than for the expanded version. Another notable feature of the graph is that retention of the pilot group dropped fastest between the first and second years. This was after they went through a largely traditional sophomore year.

Figure 5

VI. Discussion

Data from the first year pilot program was very positive. Assessment showed that it

• more than halved the attrition rate of first-year engineering students

• nearly doubled the percentage of students passing two semesters of physics on schedule

• increased the percentage of students passing calculus on schedule by 40%

• increased performance of students on common final exams in calculus by more than a grade point and a half, despite having a significantly higher percentage of students actually take the final

• produced performance improvements in physics and English relative to the traditional program.

The second year of IMPULSE saw similar results to the pilot in terms of student success and performance in calculus and physics on common exams. The areas that dropped were the first year retention rate and the percent of students passing the physics sequence on schedule.

The drops in retention rate and in physics success in the second year of IMPULSE may be attributable to changes that occurred in the expansion. We will look for assessment tools, such as exit interviews, to try to get further insight. Without further information at this time, however, we can infer that we should try to return to former practices from the pilot program where it is reasonable to do so. For example we should:

• give new IMPULSE faculty members good training in the methods, ideally also having them assist in the classroom in a prior semester

• meticulously have routine weekly meetings among professors for the same cohort to coordinate on student and team problems and discuss teaching methods and topics

• schedule a common class time among all IMPULSE cohorts for presentations from professionals, and have lots of such events

• have team building projects and special events that also have a social aspect to bond students to each other and to the program.

Our experience with IMPULSE so far indicates that an integrated first-year program can continue to work well as the novelty wears off. Nonetheless, in order for that to happen, assessment must be done and effort must be applied to maintain and improve it.

The IMPULSE program, like others that use technology in the classroom and involve integration of subjects, is a highly ordered system. It requires coordination and cooperation and has special issues of scheduling and technology support. We all know that systems will return to a lower ordered state if allowed to do so (the forces toward chaos are relentless!). We believe that the lowest ordered state at a university is when each faculty independently just lectures from old, worn notes to sleeping students. Our experience with IMPULSE so far indicates that its demonstrated performance justifies the will and effort needed to maintain it as a highly ordered program.

Acknowledgements

The authors especially wish to thank:

• The Davis Educational Foundation, the National Science Foundation and the Foundation Coalition for their support.

• Dr. Judy Sims-Knight for assistance in the assessment activity.

• The IMPULSE team for impressive effort to build an excellent program.

• Many faculty members, administrators and staff at UMD who have given generously of their time and resources to assist the development, implementation and assessment of the IMPULSE program.

N. A. PENDERGRASS
Dr. Pendergrass is presently Department Chair and Professor in the Electrical and Computer Engineering Department at the University of Massachusetts Dartmouth. He has been a leader in the development and implementation of innovative educational programs and courses and he has been instrumental in getting the external and internal funding needed to effect significant change. Dr. Pendergrass received a B.S. degree in Electrical Engineering at the University of Missouri at Rolla in1967, and M.S. in Electrical Engineering from Purdue University in 1969, and a Ph.D. in Electrical Engineering at the University of California, Berkeley, in 1975.

RAYMOND N. LAOULACHE
Dr. Raymond N. Laoulache is an Associate Professor of Mechanical Engineering at the University of Massachusetts Dartmouth. He received his Ph.D. from Brown University, his MS and BS from Northeastern University. Dr. Laoulache's research interests are in the areas of multiphase fluid flow. He developed the engineering component of the integrated Engineering curriculum (IMPULSE). Currently, he serves as the Director of the IMPULSE program.

EMILY FOWLER
Emily Fowler is Assessment Specialist at the University of Massachusetts Dartmouth's Office of Institutional Research. She received an undergraduate degree in psychology from Harvard University and a master's degree in health care administration from University of North Carolina at Chapel Hill. She now applies both qualitative and quantitative assessment methods to new initiatives and curricular changes at UMass Dartmouth's College of Engineering. She recently served as the Strategy Director for Assessment and Evaluation for the National Science Foundation's Foundation Coalition.