14 Nov 1997 ............... Length about 2700 words (17000 bytes).
This is a WWW version of a document. You may copy it.
How to refer to it.
Fetch a postscript version to print.
Niche-based Success in CAL
Stephen W. Draper
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve
This was given as a paper at CAL'97. A version appears in the
journal Computers and Education 1998 vol.30, pp.5-8.
In reflecting on which pieces of CAL stand out as strikingly successful,
this paper argues that there are no generalisations about what features of
technology or software type makes a piece of CAL successful, but that on the
contrary the most definite successes seem to come from a close fit between a
piece of courseware and its situation of use that is specific to that niche.
These are usually cases where a teacher analysed what was particularly weak in
an existing situation and thought of how technology could be used to address
that bottleneck. Often the technology is not particularly innovative, but it
is a close match to the needs of that niche. This paper develops this argument
by reference to a number of pieces of software which have little in common with
each other, but all of which have proved to promote learning powerfully.
The TILT project (Doughty et al. 1995) involved many pieces of software,
some bought in and some developed as part of the project, many of which were
evaluated in classroom use. Over 20 studies were done across diverse
departments from Accountancy to Zoology. Although most were satisfactory, in
that adequate learning was shown to occur when they were used, only a few stood
out as being associated with marked increases in effectiveness. Is it possible
to see anything in common among these successes, and can the hypothesis be
extended to other examples?
Many developments past and present of CAL have been technology driven, and many
have failed. For instance, they might be driven by a desire to make education
cheaper by replacing some of a teacher's functions by a machine; or a desire to
develop new, hopefully better, kinds of education by exploring what a machine
can do. But often projects have been led by technology: how would we "do the
teaching" by or around a computer. The worst are "how to replace human
teachers by a machine". Less extreme have been projects to explore what can be
done by technology that seems neat to the developers. But the best projects in
terms of educational benefits, I argue here, have followed an approach where
the work is driven by "what is worst about the present method of teaching, and
how could that problem be solved (possibly by technology)?".
My hypothesis is that there are no generalisations about the goodness of
CAL (any more than about the goodness of books in education) that refer to
technical features of the CAL, as opposed to features of the situation. There
are some substantial successes, but these come from a fit between the design of
a piece of CAL and a particular educational niche. There is no general recipe
for making CAL good independent of the educational problem; CAL is not a
panacea; what is good about one application is probably not what is good about
another — in the cases described below, there is little overlap between them.
On the other hand, computers are a very general tool: so general, that even
calling them a communication medium like print or film may be misleading, as
other media are basically monologue carriers that can only be used for
exposition. Computers are not only multimedia carriers, but can also be
interactive, and other things. The cases below tend to depend on different
properties of the computer from each other.
Success comes from considering a piece of teaching, a teaching problem;
identifying what is the main problem with it at present: the bottleneck
limiting its quality (effectiveness) at present; designing a way (using a
computer in these cases) to tackle that bottleneck. Note what this approach is
NOT, in contrast to many practices and assumptions. It is not: fund the
technology, to get the money think of some way to use computers; or similarly,
we want to replace lectures or teachers by computers, so implement a
replacement (generally, by imitation on computer of what a person does, or
rather what the designer thinks they do). It is not even: simulations are good
and can be done on computers and can't be done otherwise, so how can I do a
simulation in this course.
Instead the educationally successful approach is instruction led. Take a
specific teaching and learning situation; identify the main limitation in the
current delivery method; design a solution. Leave other things alone: do not
try to implement everything on computers: the best solutions are often
characterised by computer mediated education that is NOT carrying the main
exposition, and may not be the centre of attention.
A department teaching Portuguese language skills recognised that the
weakest point was getting sufficient conversation practice for the students.
It is widely accepted that second language learning is best done, not by book
study, but by conversation practice. However providing Portuguese speakers for
students to practice with is expensive: the norm was a couple of hours a week
in a class of about 20 students. This was the crucial bottleneck in delivering
good teaching and learning. Consequently software was developed to give
students conversation practice. Their contribution was audio recorded by the
software (and could be played back and modified by the students). The
conversational context was provided by pictures (e.g. a street market in
Brazil), text, and pre-recorded sound. This is not intelligent software, but
it successfully exploits available media to provide an adequate conversational
framework. The net effect is that students get many more hours a week
practice, and the learning outcomes as measured by exams and validated by an
external examiner have improved enormously. More details are given in McAteer
et al. (1996).
Traditional seminars in a music department had been abandoned as too
poor in quality. The format required one student per session to have prepared
a paper which they read to the others. A discussion was supposed to follow,
but in practice only the tutor made any contribution. Other students either
did not attend, or said nothing. One reason for this situation was that
students from different faculties and different years would be taking the same
course. A first year engineer might be sitting next to a third year music or
literature major, and naturally feel they were not in a position to debate.
Email seminars were introduced, in which both the papers and the discussion
were done by email within small groups. Six out of eight of these groups were
markedly successful by various measures (number of contributions, quality of
contribution, student opinions), in contrast to almost no success in the old
face to face format. More details are given in Duffy et al. (1995).
This success is probably due to at least the following factors. Students could
meditate and take time to formulate contributions and responses, rather than
having to think and articulate them on the spot in "real time". The tutor
mediating the email seminars, whose skill was a crucial factor, applied a
marking scheme that rewarded all contributions with bigger rewards for better
contributions (or conversely, could be thought of as penalising silence). It
would not be practicable to apply this reward scheme in a face to face seminar.
Thus while the technology was very simple, it could overcome crucial problems
in the traditional delivery that it replaced. While to some extent these
advantages would apply as an alternative to any use of face to face seminars,
this department has, as noted, a particularly unpromising situation for
fostering relaxed discussion: so the software solution matched this case better
than it might others.
If you consider the instruments a dentist uses they are mostly about the
size of a pen, and the crucial actions involve the motion of the tip, about the
size of a pen nib. Clearly it will be hard for students to see what is going
on in a demonstration unless they are very close indeed: closer than they could
be on average even in small group teaching, yet one-to-one teaching is too
expensive. This reasoning led to the development of simple animations as an
adjunct to (not replacement for) the existing practical teaching, which already
used short talks by teachers and practice on life size models of heads with the
instruments being taught. The success of this approach was established in a
series of evaluations done by Erica McAteer during the TILT project, but not
yet published externally. This is a particularly clear example of how
computers can be successfully used, not to replace aspects of teaching and
learning that already work well and are probably essential (tutors to answer
unexpected questions, equipment for personal practice) but the one aspect that
could not be delivered well (a clear view of small instruments and the motions
to be used in employing them).
It has frequently been claimed that simulations are leading examples of
how the use of computers can benefit education. They can allow students to
explore situations that too dangerous, or too expensive, or ethically
unacceptable to explore in a lab. To the extent that this is true in a
particular case, then it fits the the arguments made here of identifying a
bottleneck in current delivery and finding a way of exploiting technology to
overcome it. However it should be noted that there is a converse that applies.
If all that is done is to replace physical equipment by a computer simulation,
then the resulting teaching will inherit the old weaknesses that lab classes
frequently have: of students following a recipe to produce a predicted result
without at any time thinking at all about the concepts that the procedure is
supposed to relate to. If you stand outside a science lab. at 5pm in many
universities and ask the students as they leave what the lab was about, they
frequently cannot tell you anything about the conceptual subject of the lab.
Often the same applies if you stand outside a computer room as students leave
having used a simulation. The crucial pedagogical weakness of many labs is not
the expense or danger, but the failure to arrange for students to be
intellectually engaged with the conceptual issues, but instead to be wholly
occupied by the mechanical procedure, by following a fixed set of steps, and
getting the official "result". The solutions that work for physical labs here
need to be applied to simulations (e.g. arranging "pre-labs" to activate the
students' related conceptual knowledge before they engage in the lab.
procedures), and conversely solutions that work for simulations (cf. Milne;
1996) could be applied to physical labs without the need for computers (e.g.
working in pairs, discussion stimulated by a tutor at appropriate moments in
the session, work sheets requiring some open ended problem solving).
Simulations clearly do have a useful contribution to make, but often it is not
the physical nature of labs that was the crucial bottleneck in learning
outcomes, so simulations by themselves will not in those cases improve learning
much.
In contrast, simulations may sometimes be just what is needed. The
Laurillard (1993, p.103) model suggests that in all subjects there are two
levels whose relationship is often neglected in teaching: the formal,
conceptual level and the level of practical action and personal experience.
However in chemistry, Alec Johnstone (1991) has argued that there are three
domains that must be related: the macroscopic domain (including what chemicals
look like in the lab) of bulk properties, the formal domain expressed in
equations that typically express attributes of single molecules, and a third
domain of the spatial and temporal (dynamic) properties of molecules and how
they fit together. Many phenomena (such as why snowflakes have a 6-fold
symmetry, what goes on during melting, and so on) belong entirely to this third
domain, which however is often neglected in teaching. Roy Tasker (personal
communication and demonstration) has developed extensive animations to address
this by accurately simulating this unseeable domain. This is a rare example of
a deep pedagogical motivation for using simulation and animation, and pilot
trials suggest that it is very powerful in stimulating learners to make new and
important connections between fragments of their existing knowledge.
The hypothesis developed in this paper came originally from asking what
was distinctive about the pieces of CAL in the TILT project that, when
evaluated in use on courses, were associated not just with satisfactory
learning but with demonstrable improvements. It also seems to apply to some
other work: that by Milne and Tasker discussed above. To test it further will
require a survey of all cases that have been evaluated in classroom use and
showed not just adequate but markedly improved learning over previous teaching
delivery.
Judging by the cases considered so far, the best cases of applying CAL to
improve learning will combine: a) an identification of a real pedagogic
problem; b) a pedagogic theory of how the educational intervention is a
solution to the pedagogic problem; c) a neat bit of CAL design. This
probably has to be followed by: d) skilled administration of the teaching and
learning using the technology; e) evaluation and demonstration of the
resulting learning gains (otherwise not only would we not know about it, but it
would probably not be maintained by the department of origin, nor disseminated
to other departments and institutions). This approach can be described by the
phrase "niche based success".
The evolutionary analogy is not superficial. A species or individual is not
"adapted" by any absolute standards. Bigness or smallness is not good, neither
are other features. They are "good" i.e. adaptive and survival-promoting by
virtue of their fit to their ecological niche. If you want to be better
adapted it is no good studying anybody else's niche, nor any other niche than
the one you are in right now. It is also a long shot studying other people's
solutions, although (as convergent evolution shows) just occasionally old
solutions may be good for you too. And finally, evolutionary adaptiveness is
strictly competitive, like markets. It doesn't matter how poor your product is
as long as no-one else is making a better one. Australia's marsupials were
fully adaptive until, but only until, rival mammals were imported: then many
instantly became unfit and are now vanishing. Similarly, computer solutions in
education are only worthwhile if they are better than their competitors, but
conversely as long as they are better than the available alternatives it
doesn't matter if they are inadequate by some ideal, but unimplemented,
standard. Much of the technology driven work on CAL has at best told us what
is possible. However if you look for it there are some examples that go a
crucial further step, and demonstrate a design for fitness for pedagogic
purpose. Their success is marked by significantly improved learning in the
niche they were designed for.
Doughty,G., Arnold,S., Barr,N., Brown,M.I., Creanor,L.,
Donnelly,P.J., Draper,S.W., Duffy,C., Durndell,H., Harrison,M.,
Henderson,F.P., Jessop,A., McAteer,E., Milner,M., Neil,D.M., Pflicke,T.,
Pollock,M., Primrose,C., Richard,S., Sclater,N., Shaw,R., Tickner,S.,
Turner,I., van der Zwan,R. & Watt,H.D. (1995) Using learning
technologies: interim conclusions from the TILT project TILT project
report no.3, Robert Clark Centre, University of Glasgow
Duffy,C., Arnold,S. & Henderson,F. (1995) "NetSem - electrifying
undergraduate seminars" Active Learning, no.2, pp.42-48
CTISS Publications, University of Oxford.
Also WWW document: URL
http://www.ilt.ac.uk/public/cti/ActiveLearning/issue2/duffy/index.html
Reprinted in Musicus (CTI Centre for Music), Vol. 4, pp.25-40
Johnstone,A.H. (1991) "Why science is difficult to learn? Things are seldom
what they seem" Journal of computer assisted learning vol.7
pp.75-83.
Laurillard, D. (1993) Rethinking university teaching: A framework for the
effective use of educational technology (Routledge: London).
McAteer, E., Harland, M., and Sclater, N. (1996) "De Tudo Um Pouco - a little
bit of everything" Journal of Active Learning vol.3 pp.10-15
Also a WWW document, URL:
http://www.cti.ac.uk/publ/actlea/issue3/mcateer/index.html
Milne,J.D. (1996) "Do students learn when they use simulations?" Conference
presentation, ALT-C'96 16-18 Sept., University of Strathclyde, Glasgow, UK.
See also abstract [WWW document]
URL http://www.csv.warwick.ac.uk/alt-E/alt-C96/papers.html#five