14 Nov 1997 ............... Length about 2700 words (17000 bytes).
This is a WWW version of a document. You may copy it. How to refer to it.
Fetch a postscript version to print.

Niche-based Success in CAL

Contents (click to jump to a section)


Stephen W. Draper
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve

Preface

This was given as a paper at CAL'97. A version appears in the journal Computers and Education 1998 vol.30, pp.5-8.

Abstract

In reflecting on which pieces of CAL stand out as strikingly successful, this paper argues that there are no generalisations about what features of technology or software type makes a piece of CAL successful, but that on the contrary the most definite successes seem to come from a close fit between a piece of courseware and its situation of use that is specific to that niche. These are usually cases where a teacher analysed what was particularly weak in an existing situation and thought of how technology could be used to address that bottleneck. Often the technology is not particularly innovative, but it is a close match to the needs of that niche. This paper develops this argument by reference to a number of pieces of software which have little in common with each other, but all of which have proved to promote learning powerfully.

1. Introduction

The TILT project (Doughty et al. 1995) involved many pieces of software, some bought in and some developed as part of the project, many of which were evaluated in classroom use. Over 20 studies were done across diverse departments from Accountancy to Zoology. Although most were satisfactory, in that adequate learning was shown to occur when they were used, only a few stood out as being associated with marked increases in effectiveness. Is it possible to see anything in common among these successes, and can the hypothesis be extended to other examples?

Many developments past and present of CAL have been technology driven, and many have failed. For instance, they might be driven by a desire to make education cheaper by replacing some of a teacher's functions by a machine; or a desire to develop new, hopefully better, kinds of education by exploring what a machine can do. But often projects have been led by technology: how would we "do the teaching" by or around a computer. The worst are "how to replace human teachers by a machine". Less extreme have been projects to explore what can be done by technology that seems neat to the developers. But the best projects in terms of educational benefits, I argue here, have followed an approach where the work is driven by "what is worst about the present method of teaching, and how could that problem be solved (possibly by technology)?".

2. The hypothesis

My hypothesis is that there are no generalisations about the goodness of CAL (any more than about the goodness of books in education) that refer to technical features of the CAL, as opposed to features of the situation. There are some substantial successes, but these come from a fit between the design of a piece of CAL and a particular educational niche. There is no general recipe for making CAL good independent of the educational problem; CAL is not a panacea; what is good about one application is probably not what is good about another — in the cases described below, there is little overlap between them. On the other hand, computers are a very general tool: so general, that even calling them a communication medium like print or film may be misleading, as other media are basically monologue carriers that can only be used for exposition. Computers are not only multimedia carriers, but can also be interactive, and other things. The cases below tend to depend on different properties of the computer from each other.

Success comes from considering a piece of teaching, a teaching problem; identifying what is the main problem with it at present: the bottleneck limiting its quality (effectiveness) at present; designing a way (using a computer in these cases) to tackle that bottleneck. Note what this approach is NOT, in contrast to many practices and assumptions. It is not: fund the technology, to get the money think of some way to use computers; or similarly, we want to replace lectures or teachers by computers, so implement a replacement (generally, by imitation on computer of what a person does, or rather what the designer thinks they do). It is not even: simulations are good and can be done on computers and can't be done otherwise, so how can I do a simulation in this course.

Instead the educationally successful approach is instruction led. Take a specific teaching and learning situation; identify the main limitation in the current delivery method; design a solution. Leave other things alone: do not try to implement everything on computers: the best solutions are often characterised by computer mediated education that is NOT carrying the main exposition, and may not be the centre of attention.

3. A language learning example

A department teaching Portuguese language skills recognised that the weakest point was getting sufficient conversation practice for the students. It is widely accepted that second language learning is best done, not by book study, but by conversation practice. However providing Portuguese speakers for students to practice with is expensive: the norm was a couple of hours a week in a class of about 20 students. This was the crucial bottleneck in delivering good teaching and learning. Consequently software was developed to give students conversation practice. Their contribution was audio recorded by the software (and could be played back and modified by the students). The conversational context was provided by pictures (e.g. a street market in Brazil), text, and pre-recorded sound. This is not intelligent software, but it successfully exploits available media to provide an adequate conversational framework. The net effect is that students get many more hours a week practice, and the learning outcomes as measured by exams and validated by an external examiner have improved enormously. More details are given in McAteer et al. (1996).

4. Seminars in a music department

Traditional seminars in a music department had been abandoned as too poor in quality. The format required one student per session to have prepared a paper which they read to the others. A discussion was supposed to follow, but in practice only the tutor made any contribution. Other students either did not attend, or said nothing. One reason for this situation was that students from different faculties and different years would be taking the same course. A first year engineer might be sitting next to a third year music or literature major, and naturally feel they were not in a position to debate. Email seminars were introduced, in which both the papers and the discussion were done by email within small groups. Six out of eight of these groups were markedly successful by various measures (number of contributions, quality of contribution, student opinions), in contrast to almost no success in the old face to face format. More details are given in Duffy et al. (1995).

This success is probably due to at least the following factors. Students could meditate and take time to formulate contributions and responses, rather than having to think and articulate them on the spot in "real time". The tutor mediating the email seminars, whose skill was a crucial factor, applied a marking scheme that rewarded all contributions with bigger rewards for better contributions (or conversely, could be thought of as penalising silence). It would not be practicable to apply this reward scheme in a face to face seminar. Thus while the technology was very simple, it could overcome crucial problems in the traditional delivery that it replaced. While to some extent these advantages would apply as an alternative to any use of face to face seminars, this department has, as noted, a particularly unpromising situation for fostering relaxed discussion: so the software solution matched this case better than it might others.

5. Dentistry

If you consider the instruments a dentist uses they are mostly about the size of a pen, and the crucial actions involve the motion of the tip, about the size of a pen nib. Clearly it will be hard for students to see what is going on in a demonstration unless they are very close indeed: closer than they could be on average even in small group teaching, yet one-to-one teaching is too expensive. This reasoning led to the development of simple animations as an adjunct to (not replacement for) the existing practical teaching, which already used short talks by teachers and practice on life size models of heads with the instruments being taught. The success of this approach was established in a series of evaluations done by Erica McAteer during the TILT project, but not yet published externally. This is a particularly clear example of how computers can be successfully used, not to replace aspects of teaching and learning that already work well and are probably essential (tutors to answer unexpected questions, equipment for personal practice) but the one aspect that could not be delivered well (a clear view of small instruments and the motions to be used in employing them).

6. Simulations to replace labs

It has frequently been claimed that simulations are leading examples of how the use of computers can benefit education. They can allow students to explore situations that too dangerous, or too expensive, or ethically unacceptable to explore in a lab. To the extent that this is true in a particular case, then it fits the the arguments made here of identifying a bottleneck in current delivery and finding a way of exploiting technology to overcome it. However it should be noted that there is a converse that applies. If all that is done is to replace physical equipment by a computer simulation, then the resulting teaching will inherit the old weaknesses that lab classes frequently have: of students following a recipe to produce a predicted result without at any time thinking at all about the concepts that the procedure is supposed to relate to. If you stand outside a science lab. at 5pm in many universities and ask the students as they leave what the lab was about, they frequently cannot tell you anything about the conceptual subject of the lab. Often the same applies if you stand outside a computer room as students leave having used a simulation. The crucial pedagogical weakness of many labs is not the expense or danger, but the failure to arrange for students to be intellectually engaged with the conceptual issues, but instead to be wholly occupied by the mechanical procedure, by following a fixed set of steps, and getting the official "result". The solutions that work for physical labs here need to be applied to simulations (e.g. arranging "pre-labs" to activate the students' related conceptual knowledge before they engage in the lab. procedures), and conversely solutions that work for simulations (cf. Milne; 1996) could be applied to physical labs without the need for computers (e.g. working in pairs, discussion stimulated by a tutor at appropriate moments in the session, work sheets requiring some open ended problem solving). Simulations clearly do have a useful contribution to make, but often it is not the physical nature of labs that was the crucial bottleneck in learning outcomes, so simulations by themselves will not in those cases improve learning much.

7. Simulations to address conceptual gaps

In contrast, simulations may sometimes be just what is needed. The Laurillard (1993, p.103) model suggests that in all subjects there are two levels whose relationship is often neglected in teaching: the formal, conceptual level and the level of practical action and personal experience. However in chemistry, Alec Johnstone (1991) has argued that there are three domains that must be related: the macroscopic domain (including what chemicals look like in the lab) of bulk properties, the formal domain expressed in equations that typically express attributes of single molecules, and a third domain of the spatial and temporal (dynamic) properties of molecules and how they fit together. Many phenomena (such as why snowflakes have a 6-fold symmetry, what goes on during melting, and so on) belong entirely to this third domain, which however is often neglected in teaching. Roy Tasker (personal communication and demonstration) has developed extensive animations to address this by accurately simulating this unseeable domain. This is a rare example of a deep pedagogical motivation for using simulation and animation, and pilot trials suggest that it is very powerful in stimulating learners to make new and important connections between fragments of their existing knowledge.

8. Conclusion

The hypothesis developed in this paper came originally from asking what was distinctive about the pieces of CAL in the TILT project that, when evaluated in use on courses, were associated not just with satisfactory learning but with demonstrable improvements. It also seems to apply to some other work: that by Milne and Tasker discussed above. To test it further will require a survey of all cases that have been evaluated in classroom use and showed not just adequate but markedly improved learning over previous teaching delivery.

Judging by the cases considered so far, the best cases of applying CAL to improve learning will combine: a) an identification of a real pedagogic problem; b) a pedagogic theory of how the educational intervention is a solution to the pedagogic problem; c) a neat bit of CAL design. This probably has to be followed by: d) skilled administration of the teaching and learning using the technology; e) evaluation and demonstration of the resulting learning gains (otherwise not only would we not know about it, but it would probably not be maintained by the department of origin, nor disseminated to other departments and institutions). This approach can be described by the phrase "niche based success".

The evolutionary analogy is not superficial. A species or individual is not "adapted" by any absolute standards. Bigness or smallness is not good, neither are other features. They are "good" i.e. adaptive and survival-promoting by virtue of their fit to their ecological niche. If you want to be better adapted it is no good studying anybody else's niche, nor any other niche than the one you are in right now. It is also a long shot studying other people's solutions, although (as convergent evolution shows) just occasionally old solutions may be good for you too. And finally, evolutionary adaptiveness is strictly competitive, like markets. It doesn't matter how poor your product is as long as no-one else is making a better one. Australia's marsupials were fully adaptive until, but only until, rival mammals were imported: then many instantly became unfit and are now vanishing. Similarly, computer solutions in education are only worthwhile if they are better than their competitors, but conversely as long as they are better than the available alternatives it doesn't matter if they are inadequate by some ideal, but unimplemented, standard. Much of the technology driven work on CAL has at best told us what is possible. However if you look for it there are some examples that go a crucial further step, and demonstrate a design for fitness for pedagogic purpose. Their success is marked by significantly improved learning in the niche they were designed for.

References

Doughty,G., Arnold,S., Barr,N., Brown,M.I., Creanor,L., Donnelly,P.J., Draper,S.W., Duffy,C., Durndell,H., Harrison,M., Henderson,F.P., Jessop,A., McAteer,E., Milner,M., Neil,D.M., Pflicke,T., Pollock,M., Primrose,C., Richard,S., Sclater,N., Shaw,R., Tickner,S., Turner,I., van der Zwan,R. & Watt,H.D. (1995) Using learning technologies: interim conclusions from the TILT project TILT project report no.3, Robert Clark Centre, University of Glasgow

Duffy,C., Arnold,S. & Henderson,F. (1995) "NetSem - electrifying undergraduate seminars" Active Learning, no.2, pp.42-48 CTISS Publications, University of Oxford. Also WWW document: URL http://www.ilt.ac.uk/public/cti/ActiveLearning/issue2/duffy/index.html Reprinted in Musicus (CTI Centre for Music), Vol. 4, pp.25-40

Johnstone,A.H. (1991) "Why science is difficult to learn? Things are seldom what they seem" Journal of computer assisted learning vol.7 pp.75-83.

Laurillard, D. (1993) Rethinking university teaching: A framework for the effective use of educational technology (Routledge: London).

McAteer, E., Harland, M., and Sclater, N. (1996) "De Tudo Um Pouco - a little bit of everything" Journal of Active Learning vol.3 pp.10-15 Also a WWW document, URL: http://www.cti.ac.uk/publ/actlea/issue3/mcateer/index.html

Milne,J.D. (1996) "Do students learn when they use simulations?" Conference presentation, ALT-C'96 16-18 Sept., University of Strathclyde, Glasgow, UK. See also abstract [WWW document] URL http://www.csv.warwick.ac.uk/alt-E/alt-C96/papers.html#five