[ Teacher overview Teacher details Learner main page Course home page (sample) ]
This page describes an "ATOM": a chunk of learning about evaluating CBL (computer based learning). The centrepiece is a practical exercise, with associated reading and some support and video conferences with a remote expert: the author. This version is designed for delivery in Feb-March 1998 to students at Heriot-Watt.
The agenda and details are on a separate page.
Designing the evaluation
The central exercise is for students to design and carry out an evaluation of
a piece of learning and teaching, preferably mediated by technology.
The target agreed is the learning on this course carried out in week 6.
Students should therefore work on preparing the evaluation (this is where most
of the work is) BEFORE week 6. During week 6 they will gather data from each
other, using the prepared instruments.
How the work will be divided will be finally decided by the local teachers (Patrick and Alison). Possibilities include the class working as one big team; splitting into two, as the topic for week 6 falls into two parts; or agreeing the overall evaluation as a class, but assigning different instruments (e.g. the quiz, questionnaires etc.) to different students.
The clients in this evaluation are the teachers delivering the course (in this case Patrick and Alison), who may improve how they do that in the light of the evaluation. You will therefore be interviewing them to discover what the various learning resources used are, what their learning objectives for week 6 are, what questions they would like an evaluation to answer. (Good evaluations typically combine issues the teacher wants answered with issues the evaluators think may be important.)
An important part of most evaluations that follow the Integrative approach is
to administer a test or quiz. This usually has to be both designed and marked
by the teachers: only they have the expert knowledge for this. It is usually
a good idea if each question in the quiz corresponds to an explicit learning
objective, which similarly must be elicited from the teachers.
Executing the evaluation: gathering data
In week 6, the design of the evaluation and the instruments should all be
ready: they just need to be used. Bear in mind that you, the students, are
the subjects as well as the evaluators here. You will probably not have the
time and patience (as subjects) to fill out 14 different questionnaires.
When designing the evaluation and dividing up work, make sure you do not
overload your subjects (yourselves). If it is decided to run alternative
evaluations in parallel, then you will only get a few students to fill in each
instrument. You could of course decide as a class to share data from some
instruments (e.g. the quiz) while trying different versions of others (e.g.
interviews).
Preliminary report
In real evaluations, the report is not usually written in one go. Instead,
interpreting the results depends a lot on what the teachers say, besides
comments from others. To do this, it is important to prepare the data
gathered and show it the teachers for their comments: i.e. to write a
preliminary report with all the data but few conclusions as soon as possible.
Discussion of preliminary reports
Based on the preliminary reports, a class discussion with the teachers and
perhaps a video conference discussion with the remote expert will help you
explore the interpretations you will want to include in your final reports.
If you do not have your data in a form (such as graphs) that can be easily
presented in the discussion, you will not be able to get the teachers'
interpretations. Our experience has been that this is crucial.
[ Teacher overview Teacher details Learner main page Course home page (sample) ]