FLAG - Getting Started
Assessment within the Context of Course Development
   (Screen 3 of 4)


A Generalized Model for Course Development


Figure 1

Curriculum, Instruction, and Assessment
The three primary components of any course are the curriculum (the "content"), the instructional methods used to deliver the curriculum, and the assessment techniques with which our success in attaining our course goals is evaluated. These three components (curriculum, instruction, assessment -- CIA) are inextricably linked, and are bound together by the goals we set for the course (Figure 1).

The CIA model presented here requires that goals be formalized at the outset. Setting goals is the first and most important step in course development. Once goals are stated, we can connect our curriculum, instruction, and assessment to them. One way of beginning to think about how to align assessment with course goals is by grouping them into three broad categories: knowledge-based goals, skills-based goals, and affective goals (affective: i.e. values, attitudes, and interests). For example, a recent survey conducted by the American Astronomical Society Education Office (Brissenden, Duncan & Slater, in preparation) found that a majority of the astronomy faculty respondents (n=29) have in common the following three goals for their introductory astronomy courses:

  1. Students should understand the size, scale, and structure of the cosmos and the motions of the night sky. (knowledge-based)
  2. Students should understand the nature of science and astronomy. (knowledge-based)
  3. Students should gain an interest in studying current events in astronomy as a life-long learning activity. (affective)
In addition, one might have skills-based goals. An example of a skills-based goal might be for students to be able to set up a small telescope and point it at a given celestial body.

While the most common classroom assessment technique employed in introductory astronomy courses -- the multiple-choice exam -- tends to provide useful information about students' fact-based knowledge, it usually does not provide useful information about other types of knowledge (e.g., concept-based). These tests also rarely information about skills-based knowledge. Traditional course evaluations similarly do not generally provide useful information about changes in student values, attitudes, and interests. Thus, common assessment techniques, while providing a means for assigning grades, often do not provide us and our students with useful feedback for diagnosing student learning. We see that by grouping our course goals into knowledge-based, skills-based, and affective goals, we can more readily determine if our curriculum, instruction, and classroom assessment techniques are properly aligned with our goals.

A useful scheme for further grouping course goals into categories that can help identify appropriate classroom assessment techniques is provided by Bloom's Taxonomy of Educational Goals.

Ensuring Course Goals Are Measurable
Determining the extent to which course goals are being achieved can be hampered if goals are stated too abstractly. How does one assess whether students have "gained an understanding of the seasons", for example? To facilitate aligning our curriculum, instruction, and classroom assessment techniques with course goals, it is important to make sure that our goals are measurable. That is, course goals must be stated in concrete terms, with words that describe realistic, demonstrable student performance. The goal relating to understanding the seasons given above might be more effectively stated as, e.g., "defines seasons" or "distinguishes importance of different factors such as tilt and distance", depending upon the desired level of expertise with this particular goal. The Aligning Goals to CATs section helps you identify measurable goals.


Bloom's Taxonomy
There is more to our students' knowledge than simply being right or wrong; rather, our students possess a continuum of knowledge with varying degrees of less or more sophistication. Hence, the criteria by which we measure student success in our courses -- our choice of classroom assessment techniques -- should vary in sophistication depending on the particular concept or skill we are assessing. To fully align our classroom assessment techniques with measurable student outcomes, we need to describe these outcomes in terms of the desired levels of expertise we want our students to achieve.

One of the most widely used ways of organizing these levels of expertise is according to Bloom's Taxonomy of Educational Objectives (Bloom et al., 1994; Gronlund, 1991; Krathwohl et al., 1956). Using Bloom's Taxonomy, we can express our desired measurable student outcomes along a six-level scale of student knowledge and ability. These levels represent a range of expertise (see Tables 1-3).

Table 1: Bloom's Taxonomy of Educational Goals: Knowledge-Based
Classifications
Descriptions
Examples
1. Knowledge Recall, or recognition of terms, ideas, procedure, theories, etc. When is the first day of Spring?
2. Comprehension Translate, interpret, extrapolate, but not see full implications or transfer to other situations, closer to literal translation. What does the summer solstice represent?
3. Application Apply abstractions, general principles, or methods to specific concrete situations. What would Earth's seasons be like if its orbit was perfectly circular?
4. Analysis Separation of a complex idea into its constituent parts and an understanding of organization and relationship between the parts. Includes realizing the distinction between hypothesis and fact as well as between relevant and extraneous variables. Why are seasons reversed in the southern hemisphere?
5. Synthesis Creative, mental construction of ideas and concepts from multiple sources to form complex ideas into a new, integrated, and meaningful pattern subject to given constraints. If the longest day of the year is in June, why is the northern hemisphere hottest in August?
6. Evaluation To judge ideas or methods using external evidence or self-selected criteria, substantiated by observations or informed rationalizations. What would be the important variables for predicting seasons on a newly discovered planet?


Table 2: Bloom's Taxonomy of Educational Goals: Skills-Based
Classifications
Descriptions
Examples
1. Perception Uses sensory cues to guide actions Some of the colored samples you see will need dilution before you take their spectra. Using only observation, how will you decide which solutions might need to be diluted?
2. Set Demonstrates a readiness to take action to perform the task or objective Describe how you would go about taking the absorbance spectra of a sample of pigments?
3. Guided Response Knows steps required to complete the task or objective Determine the density of a group of sample metals with regular and irregular shapes.
4. Mechanism Performs task or objective in a somewhat confident, proficient, and habitual manner Using the procedure described below, determine the quantity of copper in your unknown ore. Report its mean value and standard deviation.
5. Complex Overt Response Performs task or objective in a confident, proficient, and habitual manner Use titration to determine the Ka for an unknown weak acid.
6. Adaptation Performs task or objective as above, but can also modify actions to account for new or problematic situations You are performing titrations on a series of unknown acids and find a variety of problems with the resulting curves, e.g. only 3.0 ml of base is required in for one acid while 75.0 ml is required in another. What can you do to get valid data for all the unknown acids?
7. Organization Creates new tasks or objectives incorporating learned ones Recall your plating and etching experiences with an aluminum substrate, Choose a different metal substrate and design a process to plate, mask, and etch so that a pattern of 4 different metals is created.


Table 3: Bloom's Taxonomy of Educational Goals: Affective
Classifications
Descriptions
Examples
1. Receiving Demonstrates a willingness to participate in the activity When I'm in class I am attentive to the instructor, take notes, etc. I do not read the newspaper instead.
2. Responding Shows interest in the objects, phenomena, or activity by seeking it out or pursuing it for pleasure I complete my homework and participate in class discussions.
3. Valuing Internalizes an appreciation for (values) the objectives, phenomena, or activity I seek out information in popular media related to my class.
4. Organization Begins to compare different values, and resolves conflicts between them to form an internally consistent system of values Some of the ideas I've learned in my class differ from my previous beliefs. How do I resolve this?
5. Characterization by a Value or Value Complex Adopts a long-term value system that is "pervasive, consistent, and predictable" I've decided to take my family on a vacation to visit some of the places I learned about in my class.


Bloom's Taxonomy is a convenient way to describe the degree to which we want our students to understand and use concepts, to demonstrate particular skills, and to have their attitudes affected. Just as it is vitally important to formalize our teaching goals, it is equally critical to assign the desired levels of student achievement to each stated goal. Though the most common form of classroom assessment -- multiple choice tests -- might be quite adequate for assessing knowledge and comprehension, this type of assessment often falls short when we want to assess our students knowledge at the higher levels of synthesis and evaluation (Bloom et al., 1994; Tobias & Raphael, 1997). We might not expect, or even desire, that our students will achieve these higher-order levels of understanding for all of our course goals, but it is important to identify to which level we do expect them to achieve.

Bloom's Taxonomy need not be applied exclusively after course goals have been defined. Indeed, Bloom's Taxonomy and the words associated with its different categories can help in the goals-defining process itself. Thus, Bloom's Taxonomy can be used in an iterative fashion to first state and then refine course goals. Bloom's Taxonomy can finally be used to identify which classroom assessment techniques are most appropriate for measuring these goals.

If assessment is to support student achievement in addition to being the process by which we assign grades, Bloom's Taxonomy provides a way to focus our instructional and assessment activities at the appropriate levels for our desired student outcomes. In addition, there are a variety of innovative strategies we can use to move curriculum, instruction, and assessment to the higher levels we often desire our students to reach. In the final section, we provide a brief description of several classroom assessment techniques designed to do just this. Links to extended descriptions and implementation procedures are included. You may also be interested in the NISE Collaborative Learning site.


How Assessment Fits Into the CIA Model of Course Development

A Roadmap for Course Development
Every SMET course has three components: curriculum, instruction, and assessment. Before we can focus on any of these components, we must (a) formalize our course goals and (b) categorize each goal via Bloom's Taxonomy. Once we have completed this task, we can focus on the curriculum and instructional methods we think will best lead to our desired student outcomes. Stopping here is not enough. We must choose appropriate classroom assessment techniques -- those techniques that are aligned with our course goals. Then we will be in the position to evaluate the worth of our curriculum and instructional methods at producing our desired goals.

The terms assessment and evaluation are often incorrectly used interchangeably. Assessment is the collecting of data to inform both the instructor and the student as to how the course is progressing or how it has ended. Assessment involves using one or more classroom assessment techniques. Evaluation is what we do with this data once we have it. Once we have acquired the assessment data, it is up to us to judge the efficacy of our instructional methods, the content of our course, and the achievement of our students.

The FLAG site provides the opportunity to investigate and clarify course goals via the section Aligning Goals CATs. Once you have clarified your goals, you will be directed to specific innovative classroom assessment techniques that are aligned with your course goals. Within each CAT you will also find information about "how to turn your data into something useful." It is this important, evaluative step that allows you to determine the extent to which you are reaching your course goals or to decide if there are changes you would like to make. This is the role of Assessment within the CIA model of course development (Figure 2).

Figure 2: Road Map of Course Development

Assessment Is Feedback for both Students and Instructors
The perspective advocated here is that we can use carefully constructed classroom assessment techniques as a means of determining whether or not we are meeting our stated course goals, in addition to assigning our students grades. For us, classroom assessment can help us answer the following questions:

  • To what extent are my students achieving the stated course goals?
  • How should I allocate class time for the current topic?
  • Can I introduce this topic in a more effective way?
  • What parts of this course are my students finding most valuable?
  • How will I change this course the next time I teach it?
  • Which grades do I assign my students?
For our students, classroom assessments answers a different set of questions:
  • Do I know what my instructor thinks is most important?
  • Am I mastering the course content?
  • How can I improve the way I study in this course?
  • What grade am I earning in this course?
Answering these questions and others can inform and improve the quality of student learning in our classes.


A Charge to Change

Over the years, we have observed that many heated discussions over assessment are actually arguments about curriculum. We can not emphasize enough how important it is to actually write down your course goals and share them with your students. Once your course goals are set, questions about instruction, assessment, and grading will be much more focused. This is a small step beyond the assessment strategies that most faculty are already doing; yet with a small investment in planning, the data acquired can provide valuable information about improving the quality of student learning.


Resources


Tell me more about Assessment: