NISE Formative Evaluation

Team Leader: Susan B. Millar

Project goal: The objectives of the NISE Formative Evaluation Team have evolved over the course of the NISE's history. The first objective was to provide the NISE leadership with immediate information about the development of the Institute as an organization to help the NISE make mid-course corrections and thus maximize its goal achievement. By the middle of year 3, the NISE Team Leaders agreed that the NISE had evolved to the point where the value of additional formative evaluation had reached the point of diminishing returns. The formative evaluation objective was completed with the production of a paper synthesizing the key themes that emerged from the formative evaluation work.

A second objective of the team was to provide formative and summative evaluation of the NISE Forums. The team pursued this objective by evaluating the First, Second and Third Annual Forums, and the Graduate Education Forum. The prototype for Forum evaluation having been established, the NISE Central Office assumed responsibility for this activity as of year 4. During year 2, the team began pursuing a third objective, which was to draw on its social science expertise to produce syntheses and proceedings of the Second and Third Annual Forums, and the Graduate Education Forum. During year 5, the team has shifted to a fourth objective—conducting an impact study of   the College Level One Team's Field-tested Learning Assessment Guide (FLAG). 

Products: The formative feedback that the Formative Evaluation Team provided during the first three years helped Institute leaders understand how various NISE constituents were experiencing and viewing the NISE teams and the NISE overall. The Advisory Team, Team Leaders, and most team members used these reports to make organizational improvements. The value of this formative evaluation process is described in "Formative Evaluation: A Key to Fostering a Cross-Disciplinary Knowledge-Building Community of Practice within a Research University." This paper explores the processes and emerging principles by which one complex, cross-disciplinary organization—the NISE—developed. In addition, it locates the development of the NISE within the context of the research literature on other cross-disciplinary organizations, and makes a case for the utility of formative evaluation for the development of academic centers.