Last changed 9 June 2002 ............... Length about 4,300 words (30,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/ilig/handsets.html. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [EVSmain] [this page]

Electronically enhanced classroom interaction

(written by Steve Draper, as part of the Interactive Lectures website)

By Stephen W. Draper, Department of Psychology,
Julie Cargill, and Quintin Cutts, Department of Computing Science,
University of Glasgow.

This is a version of a paper given at ASCILITE2001; and published as:
Draper,S.W., Cargill,J., & Cutts,Q. (2002) "Electronically enhanced classroom interaction" Australian journal of educational technology vol.18 no.1 pp.13-23.

Abstract

A design rationale for introducing electronic equipment for student interaction in lecture theatres is presented, linking the instructional design to theory. The effectiveness of the equipment for learning depends mostly on what pedagogic method is employed: various alternative types are introduced. Prospective studies are outlined for exploring its use over new ranges of application. Rival views of the concept of interactivity are one way to organise the evalution of this learning technology.

Introduction: the design

This paper describes the design rationale for introducing electronic equipment for student interaction in lecture theatres, and the studies now in prospect of the use of this equipment.

The equipment is essentially that of the TV show "Who wants to be a millionaire?": every member of the audience (i.e. each learner in a lecture theatre) has a handset similar to that of a TV remote control, the presenter displays a multiple choice question (MCQ), each learner transmits the digit corresponding to their chosen answer by infrared, a small PC (e.g. a laptop) accumulates the answers, and it displays, via the room's projection system, a bar chart representing the distribution (totals) of the responses to audience and presenter alike.

This may be called (following Michael McCabe) a "Group Response" (GR) system. Its essential feature is that, regardless of group size, both audience and presenter get to know the distribution of responses (alternatives chosen), and how their own personal response relates to that distribution, but however without knowing who chose what. This means everyone contributes, and the representativeness of each response is also exactly known. On the other hand, the privacy of the choice means that, unlike in face to face groups, each individual can express the choice they incline to, rather than only a choice they feel able to explain and justify to others. These are quite often different both in science learning and in social processes.

The main pedagogic categories of use of the equipment are:

Assessment, both formative and as practice for summative assessment. Here the MCQs are meant to test content knowledge, and perhaps are drawn from a bank used for formal assessment on the course. The advantages of the equipment here are that "marking" is fully automatic, each learner can know immediately if they gave the right or wrong answer, how their performance on the question compares to the group as a whole, tailored explanations may be given by the presenter, and the presenter equally sees immediately how well the class measures up on that question (feedback from learners to teacher). The feedback cycle here takes about two minutes per item (somewhat longer if explanations are given). Any kind of MCQ may be used, provided the response is a single selection from a small fixed set: whether the usual rather shallow item, or one designed to probe understanding more than information retention (possibly by prior use of phenomenography (Marton, 1981; Marton & Booth, 1997) to map the common misconceptions).
Formative feedback on learning within a class (i.e. within a contact period). Similar items might be used, but in order to discover and demonstrate what points should be focussed on during the class. Thus one or several such question items at the start of a class could be used to select a topic for detailed coverage, while the same or similar items at the end could demonstrate to what degree the group now understood the topic.
Formative feedback to the teacher on the teaching i.e. "course feedback". While the standard questionnaire at the end of a term, semester, or course has in general only a small effect on changing anything (Cohen, 1980) and takes a year to do so, a quick on the spot anonymous poll half way through a class (e.g. on whether the pace is too fast or too slow, the jokes too numerous or infrequent, the examples too many or few) can be used to change things immediately. Making adjustments to the teaching every 30 minutes, instead of only once a year, and furthermore making them for the particular group that gave the feedback, is much more likely to be effective than the usual practice.
Even better on the spot evaluation might be done by asking students what the best and worst issues are in the teaching at present. Assuming that even a handful are willing to mention an issue to the teacher's face, these can then be put as questions to the class, and an accurate secret ballot taken on the breadth of support for each one. This cycle of an open-ended evaluation probe, followed by systematic (and quantitative) measures of the issues thus identified, is the best evaluation practice: much better than using standard course questionnaires for all classes, learners, teachers, and contexts. Normally it would take days or weeks: but the whole 2-phase cycle could be done within 10 minutes.
Peer assessment could be done on the spot, saving the teacher administrative time and giving the learner much more rapid, though public, feedback. For example if each student has to give a verbal presentation and this is peer assessed, then at the end of their talk the teacher can display (say) each of 10 criteria in turn, and get the other students to enter their mark for this anonymously but on the spot, with the totals displayed.
Community mutual awareness building. At the start of any group e.g. a research symposium or the first meeting of a new class, the equipment gives a convenient way to create some mutual awareness of the group as a whole by displaying personal questions and having the distribution of responses displayed. For example, at a research meeting start by asking people's ages (which illustrates the advantage of anonymity), and the kind of department or institution they come from, and some alternative reasons for attending. At the start of a class, I might ask whether each student is straight from school or not, their gender, which faculty they belong to, whether they signed up for the course because it is their main interest, a side interest, or are just making up the number of courses they do.
Experiments using human responses: for topics that concern human responses, a very considerable range of experiments can be directly demonstrated using the audience as participants. For instance visual illusions may be displayed and the equipment used to show what degree of uniformity of response is found. Priming effects can be shown, where the perception of an ambiguous word or display is affected by what was shown before. The performance of witnesses to a crime (including the effects of some well known biasses) can be explored by showing a short film, followed by various questions about what was shown. Social psychology effects, e.g. on conformity, could be demonstrated if responses to early questions were faked to see whether the class then changed their responses to later questions. In general, experiments that rely only on a stimulus and a forced choice response, but not on accurate measurements of reaction times, can usually be demonstrated in this way. Thus for the particular case of psychology, but also for parts of physiology, medicine, economics, and so on, direct demonstrations of relevant effects can be mounted.
Possibly the most productive application, however, and the one with the largest body of existing research, is in using the equipment to initiate a discussion. Here, a carefully chosen MCQ is displayed and the learners register an answer, thus privately committing to a definite opinion. The presenter then, however, does not indicate the "right" answer but directs the class to discuss their answers with each other. Having to produce explanations and reasons is powerfully "mathemagenic" (conducive to learning), which of course is why researchers learn so much from giving talks and writing papers, and why teachers make their students write essays and answer questions. The equipment can be a significant help in introducing this, even into large classes.
This method of teaching by questions has been widely used and researched, although mostly without electronic aids (Hake, 1998a, 1998b).

Justification or design rationale

Although techno-enthusiasts, and indeed many government agencies or departments, have been pushing the use of computers and other technologies in education, and there are now many people whose job is essentially this and who are therefore necessarily aligned with this indiscriminately positive attitude, there is still very little good evidence of benefits. Perhaps this is not surprising: Landauer (1995) found it very hard to discover evidence of economic benefits for using computer technology in general. Besides suggesting that developing evaluation methods powerful enough to test this may be a more important, if more difficult, research task than generating yet another application of technology to learning, this does mean that each application should be carefully justified. In a review of a number of applications (Draper 1998), I argued that most applications showed no significant improvements over what they replaced, but that the few striking positive exceptions were characterised by "niche-based design": by a good fit between a particular learning situation and a specific technical solution. They were projects that had been inspired by identifying a specific weakness in current delivery, and had focussed technology on solving that problem rather than on replacing what had been adequately done before. Can the use of the classroom equipment described above meet the implied standard of justification?

In considering large classes in large lecture theatres, the main problem is usually analysed as to do with the lack of interaction and the consequent extreme passivity imposed on the audience. In terms of Laurillard's model of the learning and teaching process (Laurillard, 1993, p.103), this situation fails to support the iterative interaction between learner and teacher that is one of her underlying principles, and more specifically does not support even activity 2: the "re-expression" by the learner of what the teacher has expressed. (This can be seen as corresponding to the constructivist requirement that learners acquire knowledge by rebuilding it on their own personal, mental foundations. Redescribing it in their own terms is an activity that powerfully promotes this.) Actually, with highly skilled learners and a teacher reasonably in tune with the group, this can nevertheless take place: for instance, where the learners take notes that are not mere dictation, but substantial re-formulations of what is being talked about. (This is a reasonable theoretical analysis of the considerable benefits I have often obtained from listening to talks at conferences where I have not asked questions, but have nevertheless learned something useful.) However this degree of skilled, silent interaction is not often present in undergraduate teaching, and large numbers usually prevent learners asking sufficient questions to repair the attunement between speaker and audience, from both a pragmatic (there isn't time for many people to ask questions) and a social (it just feels too embarrassing) viewpoint.

That, then, is the diagnosis offered here of the chief weakness of lecturing to large groups. The handsets and associated equipment offer a way of tackling that weakness by (a) allowing each learner independently to generate an answer (at least a partial instantiation of activity 2), whereas otherwise only the handful who put their hands up really do this; and (b) to register that answer and so maintain the motivation for doing it; and in so doing (c) to affect the course of what happens next. This contingency (dependence of the teacher's behaviour on what the learners do) is true interactivity: one of the underlying principles of Laurillard's model, represented there by the to and fro repetition of activities between learner and teacher. The summed responses are real feedback to the teacher, that naturally leads to adjustments and reattunement if required, and in fact do this better than questions and answers from any subset of individuals. Furthermore the equipment offers an anonymity of response that addresses the shyness that additionally inhibits any interaction.

As mentioned in passing, there are some other reasons for expecting benefits with the types of pedagogic use other than initiating discussions. Formative, summative, and peer assessment could be made more convenient and quicker (and so more affordable for both learners and teachers in terms of time). Starting to build a sense of a learning community could get off to a quicker start, especially in large groups. Demonstrating experimental effects instantly connects the abstract overview given to a personal perception and experience of it: something very helpful to learning both for retention, comprehension, and for a fuller content of learning. The biggest learning gains, however, are likely to come from the much better and quicker feedback from learners to teachers, allowing better attunement of the delivery; and from the method of teaching by questions i.e. of discussions in class (whether in small groups, plenaries, or a combination) initiated by well designed questions and by getting each individual to start by committing to an initial position.

Is the equipment really likely to be any better than the alternatives? The simplest alternative is getting students to give a show of hands. This equipment crucially offers more privacy (it's a secret ballot, and important for just the same reasons). Other rival technologies are to issue each student with a cardboard or plastic cube with a different colour on each face, to be turned to show their "vote"; or with a large sheet of paper divided into a few squares each with a digit in, that the student can hold up in front of their bodies and point to the digit they select. These methods allow only near neighbours to see a student's selection. Thus the electronic equipment offers somewhat better privacy, but the difference may only be crucial with new classes: it is quite possible that with a class grown comfortable with the electronic version, moving over to a cheaper but less private version might not destroy the interactivity. The electronic version also provides faster and more accurate counting of the results: most presenters will only estimate shows of hands to about the nearest 20%, unless they have the patience to pursue and count exactly even with large groups. The accuracy may have a small but not negligible value in making all participants feel their views count, and are not just lost in crudely approximate estimates.

In scrutinising this instructional design rationale, note that it does not feature computers in a starring role (although actually one is crucial to tabulate the results): the instructional design mostly isn't in the equipment or software, but in how each teacher uses it. That is a lesson which perhaps the rest of the learning technology field should take more to heart if the aim is in fact to improve learning rather than to promote the glamour of machines. On the other hand, note too that this design does not fit with a simplistic interpretation of the slogan "learner-centered". Improved learning and the learners are the ultimate intended beneficiaries, but one of the important ways that end is achieved is by first serving the teachers better, by giving them much better, faster, and more detailed information on what the learners are thinking now, and where their problems are at each point.

Prospective explorations

There is a considerable history and community of practice in using such equipment in the specific area of promoting discussion (the last of the pedagogic uses listed above) and so improving student understanding in science and engineering at the school and early university levels (e.g. Hake 1998a, 1998b). The authors have obtained sufficient equipment for several lecture theatres, and are about to begin exploratory studies, particularly with a view to exploring the range of applications, and how far its utility can be demonstrated beyond its best established application area. We hope to trial its use in all of the pedagogic modes listed in the first section, in two universities (Glasgow and Strathclyde), in at least two disciplines (psychology and computer science) in both universities together with several others as opportunities arise, at various levels (years) in undergraduate programmes, and in a range of group sizes from 300 students downwards. (The biggest need and the biggest potential gains are in the largest group sizes, but innovation is of course a lot "safer", i.e. easier to manage, in smaller groups.)

The exploratory studies should yield practical knowledge such as question banks for the participating disciplines, and how much support is needed for first time use (a new lecturer and students who haven't used the equipment) and for regular use. They will also yield evaluation results on what benefits can be demonstrated. We hope to use a version of the method of Integrative Evaluation (Draper et al. 1996) to address both these aspects.

Interactivity

According to Jim Boyle (personal communication), students are generally, although not universally, enthusiastic about this approach, even over long periods (e.g. regular use throughout a year). When asked if they regard the interactive equipment as an advantage or not, classes typically show a spread of opinion such as 70% for it, 20% indifferent, 10% definitely opposed to it. Investigating more deeply than general student preferences will require more, and more sophisticated, measures.

Some of the most important evaluation issues can be organised around the notion of interactivity. Some researchers tend to an almost mechanical interpretation of interactivity e.g. counting the number and branching ratio of choice paths for users in multimedia learning software (Sims, 1997; Hoyet, 2000). With this equipment, that corresponds to the number of questions put to the learners for them to respond to, regardless of their content. It also corresponds to the effects we may well see of novelty, of the perception that the teachers are taking special trouble over the teaching (the Hawthorne effect; Mayo, 1933), or simply of physiological arousal (the physical activity involved in pressing buttons i.e. mechanical interactivity) which has led to the heuristic rule of not lecturing for more than 20 minutes without a pause, having the audience move around periodically, etc. On the other hand, if we believe in the Laurillard model, then the important factor would probably be the amount of time each learner spends on activity 2 ("re-expression"): so using the handsets should be better than a non-interactive monologue, but not as good as time spent in peer discussion (open-ended verbal responses rather than selecting one of the digits on the handsets). In other words, the measure of it would be the number of mental and verbal responses a learner makes (in discussion) rather than the number of button presses on the handset. On the other hand again, if what is important about "interactivity" is actually changing what happens by visibly affecting the teacher (i.e. genuine human-human interaction with the actions of one party being contingent on those of the other), then it will be changes to what the session is used for as a result of responses to questions near the start that predict the largest learning gains. Varying approaches in classes, and taking independent measures both of learning and of enjoyment or alertness should eventually allow such questions to be decided. Measures taken over time (e.g. weeks) should allow any halo and Hawthorne effects to be independently identified, if they are present, with enthusiasm decaying as the novelty wears off, or performance being independent of the learning activity tried and only dependent on the perceived interest of the researchers.

Other technical details

There are some further detailed issues that arise, and could be investigated. The particular equipment used transmits not only a digit to signal the learner's selected response to the question, but also a confidence level (high, medium, or low), and an ID for that handset which may or may not have been arranged with a known mapping to the student's identity. Furthermore the number of attempts each learner makes at the question before the cutoff time may be recorded. The GRUMPS (2001) project is interested in exploring data mining of records of such student interactions, though that involves negotiating issues of privacy and data protection with the students. We are writing software to smooth the integration of the equipment with other lecture facilities (e.g. the use of powerpoint presentations), and with keeping records of the interactions.

There seem to have been a variety of particular equipment used in the past, and more than one type is currently available. For instance a one-button system has been used (Massen et al., 1998), though that required each response option for a question to be attended to separately. Various numbers of buttons are offered in other equipment, and sometimes the ability to enter multi-digit responses and transmit them as one number. Wired, radio, and infrared implementations have been used. Currently infrared proves cheapest. Already technically feasible, though not yet financially attractive, is the solution of equipping every student with a radio-linked PDA (e.g. palmtop computer). Functionally, the features that can matter to further pedagogical tactics include: entering multidigit numbers (e.g. to identify the student), entering a sequence of digits to specify a sequence or set of response options rather than exactly one as an answer, and free text entry. When the latter becomes widely available, we can at last address a fundamental problem of discussion groups (such as research seminars) where many people want to ask a question: which is the best question to take for the group as a whole? Using only voice, we cannot know what the set of candidate questions is without having them asked. With textual group responses, everyone's questions could appear in front of the speaker and/or facilitator, and could then be grouped, sequenced, and sorted by priority. Meanwhile, as the technology (especially radio communication techniques) advance rapidly, we can focus on how we would use additional functions, and what their pedagogic rationale is.

Conclusion

The studies in prospect with this equipment should eventually allow us to pronounce on the validity of the design rationales presented in this paper. These studies will use measures of learning outcome, attitudes, and engagement as dependent (i.e. output) variables. They may range over, as independent (i.e. input) variables, two or more universities, three or more levels of university class and so student experience, two or more academic subjects, class sizes up to 300, and all the pedagogic strategy types described above.

Acknowledgements

This work is being supported in part by the EPSRC funded grant to GRUMPS (GR/N38114), and also in large part by the University of Glasgow, both directly and through the TLC project. Many thanks to Prof. Jim Boyle of Strathclyde University, whose rich existing experience with this equipment gives us confidence in proceeding, and many practical tips. We look forward to collaboration with him, and with David Nicol who has done a pioneering evaluation of Jim's work. Thanks also to Michael McCabe, Mae McSporran, and the anonymous referees for points that have been incorporated into this version.

References

Cohen,P.A. (1980) "Effectiveness of student-rating feedback for improving college instruction: a meta-analysis" Research in Higher Education vol.13 pp.321-341

Draper, S.W. (1998) "Niche-based success in CAL" Computers and Education vol.30, pp.5-8

Draper,S.W., Brown, M.I., Henderson,F.P. & McAteer,E. (1996) "Integrative evaluation: an emerging role for classroom studies of CAL" Computers and Education vol.26 no.1-3, pp.17-32

GRUMPS (2001, May 30). The GRUMPS research project [WWW document]. URL http://grumps.dcs.gla.ac.uk/ (visited 2001 June 1)

Hake,R.R. (1998a) "Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses" Am.J.Physics vol.66 no.1 pp.64-74

Hake,R.R. (1998b) "Interactive-engagement methods in introductory mechanics courses" submitted to J.of Physics Education Research

Hoyet,H. (2000) "Graphing Interactivity in technology-based training" TechTrends vol.44 no.5 pp.26-31.

Landauer,T.K. (1995) The trouble with computers: Usefulness, usability, and productivity (MIT press; Cambridge, MA)

Laurillard, D. (1993) Rethinking university teaching: A framework for the effective use of educational technology (Routledge: London) p.103. A diagram of her model

Marton,F. (1981) "Phenomenography -- describing conceptions of the world around us" Instructional Science vol.10 pp.177-200.

Marton,F. & Booth,S. (1997) Learning and awareness (Mahwah, New Jersey: Lawrence Erlbaum Associates)

Massen, C., Poulis, J., Robens, E., Gilbert, M., (1998) "Physics lecturing with audience paced feedback" American Journal of Physics Vol.66.

Mayo,E. (1933) The human problems of an industrial civilization (New York: MacMillan) ch.3.

Sims, R. (1997) "Interactivity: A Forgotten Art?" Computers in Human Behavior vol.13 no.2 pp.157-180.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [EVSmain] [this page]
[Top of this page]