Occasional Paper

The Role of Formative Evaluation in the Development of an Interdisciplinary
Academic Center

Susan B. Millar

Executive Summary

This Occasional Paper illustrates how formative evaluation helped the leaders of an interdisciplinary academic center—the National Institute for Science Education (NISE)— not only achieve a number of their original stated goals, but also develop from an amorphous group into an institute with many truly shared and well-defined goals, intentional organizational practices, and a distinctive culture. Evaluators played a critical role by providing information and heuristic models that helped the NISE reflect on itself and make numerous pragmatic improvements in organizational practices. More importantly, the process of working with the evaluators encouraged within the leadership a questioning and problem-solving organizational style that enabled them to become more reflective practitioners of the art of developing a productive interdisciplinary organization.

The paper describes the formative evaluation processes used and the range of outcomes that resulted from these processes. Examples of the outcomes—from the specific and pragmatic to the conceptual and broad—are that formative evaluation helped the leaders

·        redefine their goal for including individuals from heterogeneous backgrounds;

·        redistribute their budget to fund fewer people for a higher percentage of their time;

·        more fully appreciate the difficulty and value of genuine interdisciplinary work;

·        improve the quality of cross-team communication processes by instituting retreats and a new members’ orientation packet; and

·        continuously see themselves anew, as a result of being invited to view themselves through a series of heuristic lenses—including “goals, strategies and outcomes,” “tightly and loosely coupled systems,” “forming, storming, norming, performing,” and “a knowledge building entity.”

The paper suggests that other groups seeking benefits of this type should consider working with an evaluator. The saga that unfolds for other groups, working with other evaluators, will surely be different from the one told here. However, formative evaluation should, at a minimum, provide the following benefits to any emerging organization.

·        The experience of being interviewed by evaluators should help participants articulate their reasons for getting involved, the goals they hoped to achieve through their participation, and the relationships between these goals and the methods they were using to achieve them.

·        Formative feedback documents and other “feedback” interactions with the evaluators should     

provide the organization’s leaders with a synthesized view of the ideas and experiences of the various participants, and enable the leaders to perceive patterns and themes in the way the organization is taking shape; 

highlight situations where the goals held by the leaders and the participants are or are not aligned, and where implementation strategies are or are not adequate for achieving the stated goals, thus enabling the leaders to make effective midcourse corrections; 

      invite the clients to view taken-for-granted organizational events and processes in the new light cast by carefully chosen or heuristic models, thereby fostering cognitive conflict for the clients, and helping them notice practices and beliefs that inhibit achievement of their envisioned goals; and     

describe and help the leaders understand the character of the emerging organization.

In short, formative evaluation should provide ideas, information, models, and language that an emerging interdisciplinary organization can use to reframe and revision what it is accomplishing and what kind of organization it is becoming. It should enhance the leaders’ capacity to function as reflective practitioners by assisting them in tacking dialectically between their most local of local detail and their most global of global structures.

PDF Full Text