Go to Introduction Go to Quick Looks Go to Conversations Go to Cases Go to Resources




Go to resource overview
Go to assessment tools for technology
Go to assessment introduction
Go to why do a study
Go to directions
Go to Activities and formal study
Go to Three types of formal studies - summary
Go to Formal studies: scanning studies
Go to Formal studies: proof (cause and effect) and tracking studies
Go to Foirmal studies: diagnostic studies
Go to Some quick and dirty advice
Go to helpful people near home
Go to quicklinks
Go to glossary of computer terms
Go to articles and funding sources

Go to previous page Assessment tools - Proof-Tracking Studies: Activities Go to next page

Introduction || Activities || Example Proof-Tracking Study || Student Survey


In the activities section earlier of this tutorial we introduced the concept of a triad:

  1. technology
  2. the activity for which it's used, and
  3. the outcomes of the activity.

Any case as normal (complex) as the algebra course at the University of Houston, Downtown, has many potential triads that each could provide the focus for a proof-tracking study. Here are just a few candidates that I've drawn from a quick reading of the Houston case. Because I'd like to demonstrate how to collect proof about whether technology helps improve student learning, I focused my attention on the subsection of the Houston case entitled "The Learning Environment."

Triad # Technology Activity(s) Outcome
I. Computer and calculator 1. Reduce math errors, increase calculation speed, increase speed of creating graphs, and thereby
2. Increase student time spent hearing about, and discussing, the nature of functions
Better, lasting student understanding of algebra concepts
II. Graphing software on the instructor's computer; projection equipment 1. Faculty can easily display graphs of varying representations and values of the function, to
2. Help students visualize mathematical ideas in the classroom, and
3. Engage students' interest so they put more attention and time into the course
Better, lasting student understanding of algebra concepts
III. Graphing software on the instructor's computer; projection equipment 1. Faculty can easily display graphs of varying representations and values of the function, and then
2. Ask students to explain their thinking about the function
Better, lasting student understanding of algebra concepts
IV. Graphing software on student computer in lab Student explores properties, graphs of functions Better, lasting student understanding of algebra concepts
V. Software, data available on or through lab computers for doing calculations that enable students to work on more complex, realistic problems than they could handle with paper and pencil 1. Students are given more complex real world issues that they translate into mathematical problems and solve
2. When discussing these issues and problems it's more likely that there will be important conversations (faculty-student, student-student) that can help students learn
Better, lasting student understanding of algebra concepts

Several of the triads have more than one activity. In most cells, the first activity is intended to encourage or support the second activity; both activities contribute to the outcome.

Think of each triad as a hypothesis about how this course, or a set of sections of the course, are intended to work. The "proof" part of the study checks on questions such as these:

  1. Is the technology in the triad being used at all?

  2. Is it being used for this activity?

  3. If there is a sequence of activities, are they all happening?

  4. What's the state of the activity? For example, in triad I, is it true that students who use computers are spending less time on calculations and drawing graphs than they would have using paper and pencil? If so, is it true that they used that 'saved time' to talk and think about the meaning of the function? Or are they doing problems as thoughtfully or thoughtlessly as before, just faster?

  5. What's the state of the outcome: their ability to understand and use functions and graphs?

  6. Is there a relationship between the extent or quality of each activity and the extent to which that outcome is achieved? For example, in triad I, if you can look at more than one section (simultaneously or over a period of years), is there a relationship between time spent talking and thinking about the meaning of the function and the scores of students in those sections on good tests of conceptual understanding? Do students who score higher on the tests also report spending more time in class and at home thinking about the functions? Do they value the technology for giving them the time to do so?

Focusing
Every question we'd like to ask about what faculty and students are doing in a course, and about what students and faculty are learning from the course, takes time, energy, and (often) money to answer. The trick is to ask as few questions as possible, as cheaply as possible, while getting the most valuable answers.

That suggests several criteria to use in translating some or all of the triads above into a study design:

  • Which activities are mostly likely to contribute to substantial improvements in the outcome?

  • If this is an investigation of a course by a faculty member and if the faculty member is the user of the information, an investigation may well be more valuable if it illuminates important phenomena that the faculty member wouldn't otherwise observe. To put it another way, studies are most likely to be valuable when their findings provide direction and energy for making decisions, usually by reducing uncertainty. (This criterion often points my attention to learning activities by students, because the faculty member can't always see what students are doing and thinking, even when they are in the classroom).

  • Where would the findings be most valuable? That often depends on who can take action and what they're feeling anxious and uncertain about. The Houston case study itself isn't very informative on this point, but it does hint that student teamwork is an area of concern. If so, a study of teamwork might be more likely to draw attention and action than a study of something else where faculty are already comfortable that they know what is happening and what to do.

Keep in mind that what you're reading is a quick sketch, not a real study. In reality, I'd have talked with the faculty and administrators in order to work out a plan of inquiry together. However, because this is simply an exercise whose purpose is to help you think about your own program, I haven't actually talked with the people in Houston.

So, pretending that I've talked with the faculty and that these are our joint decisions, here they are:

  1. I'm pretending that the Houston people have told me that the board and future funders want some evidence about whether the technology has been of value in improving algebra learning before making a decision on funding for more equipment. These folks are neither true believers nor hostile to begin with; they'd just like evidence that could help them make up their minds.

  2. We decided to focus on the activities in triads III, IV, and V. Each one is closely related to the technology, involves student activities, and seem to the faculty to have special promise for improving learning outcomes.

  3. For outcomes we decided to assess student skills not only at the end of the term but, even more important, in later courses. After checking the FLAG web site and finding little of direct relevance to functions and algebra, we decided to ask our colleagues teaching later courses in which these students enrolled to come up with a test of their understanding of functions and graphs that could be used as a diagnostic early in their classes, and which could also feed us information about how our students were doing. Our interest was in the adequacy of their lasting, useable understanding of functions. That news, even if good, wouldn't tell us much directly about the value of the technology, however.

  4. So, for studying activities, we decided to focus on whether technology was actually being used in ways of great value to:

    • Encourage students to react to functions and graphs displayed in class. Were we and our colleagues really taking consistent advantage of the power of the technology to stimulate a "what-if" dialogue with, and among, our students. "What if we were to change this parameter - how would the graph change?" "Here's a graph. What function could produce it?"

    • Help students explore the characteristics of functions and their graphs while working in the lab

    • Introduce problems that are a) more complex than before, b) more exciting for students, c) able to help students see functions in action.

To sum up, here are the triads we decided to study over multiple sections and over several years to see if we could provide evidence that technology was being used to gradually improve algebra learning. (Note that these are the same triads from up above.) The boldfaced words and phrases indicate components of the activities where we'll focus our data gathering.

Triad # Technology Activity(s) Outcome
III. Graphing software on the instructor's computer; projection equipment 1. Faculty can easily display graphs of varying representations and values of the function, and then
2. Ask students to explain their thinking about the function
Better, lasting student understanding of algebra concepts, as measured toward the end of the term and in later classes requiring an understanding of functions
IV. Graphing software on student computer in lab Student explores properties, graphs of functions
V. Software, data available on or through lab computers for doing calculations that enable students to work on more complex, realistic problems than they could handle with paper and pencil 1. Students are given more complex real world issues that they translate into mathematical problems and solve
2. When discussing these issues and problems it's more likely that there will be important conversations (faculty-student, student-student) that can help students learn

The next step is to use this rather complex triad to create suggestions for the kind of data that could help prove whether technology is in fact helping students learn algebra by these mechanisms. The same data could be useful in tracking whether the program's use of these technologies and activities is becoming more effective in fostering understanding of functions over a period of years.

Introduction || Activities || Example Proof-Tracking Study || Student Survey

Go to previous page Go to next page


Introduction || Quick Looks || Conversations || Case Studies || Resources

Search || Who We Are || Site Map || Meet the CL-1 Team || WebMaster || Copyright || Download
College Level One (CL-1) Home || Collaborative Learning || FLAG || Learning Through Technology || NISE