Teaching collaboration: how we made it work

Contents (click to jump to a section)

• Preface
  • Acronyms
• Introduction 1: nature/status of this document
• Introduction 2: Context of the MANTCHI project
• Introduction 3: ped-led issue
• How could we have done it?
• How did we do it?
  • 1. Offline attempts
  • 2. Actual collaborations before the meeting
  • 3. The meeting (Project meeting)
• How was it?
• Possible types of cross-HEI collaboration
• Pedagogy-led vs. technology-led planning
  • My own theoretical slant
  • Is this a fair view?
    • Aversion to educational prescriptive pronouncements
    • Tacit expressions of pedagogical concern
    • Spending meeting time discussing technical (software) issues
    • Urgency
    • "Social dynamics": how the meeting was organised
    • Was it really ped-led?
    • How I'd like to spend meetings
  • What's all this got to do with the project aims?
  • Conclusion on the ped-led issue
• Dropping topics to make room for collaboration
  • Against change
  • In favour of taking help (collaboration)
  • My reluctance
• Proposed future recipes
  • Recipe 1
  • Recipe 2 (Wider scope)
• Possible project top level conclusions
• Summary
• References

A Technical Report
by
Stephen W. Draper
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve

Preface

This note stems from the project meeting 27 May 1997.

Acronyms

• ATOM = Autonomous Teaching Object for/in Mantchi. (This is currently ambiguous between all units of teaching collaboration and the specific recipe of one content-specific student task.)
  
• HEI = Higher Education Institution (~university).
  
• AG = Answer Garden: used here as a generic term for any software inspired by the stream of work stemming from the original answer garden.
  
• MANTCHI = may be an acronym, but just take it as the name of our project.
  
• "ped-led" = pedagogically driven decision making.

Introduction 1: nature/status of this document

This report is based firstly on my own views and perceptions as a project participant; secondly my on talking to other project members (PDG, ACK, Terry Mayes, David Benyon, Julian); and thirdly on Margaret Brown talking to three other participants at the workshop who were HCI teachers but not MANTCHI members.

Introduction 2: Context of the MANTCHI project

Introduction 3: ped-led issue

However the distinctive feature and strength of the project is the assembly of a collaboration of many teaching academics: cross HEI, all teaching one subject (HCI).

To satisfy the project promises we have to: a) collaborate across HEIs in our teaching, b) use AnswerGarden (AG) software somehow in that teaching.

The problem, as I now see it, is that:

• If you want to demonstrate technology use, then you will plan from the technology. (But this has almost always led to indifferent quality of learning.)
  
• If you want to demonstrate improved learning, then you need to plan from the learning needs. (But this may not need technology or collaboration.)
  
• If you want to demonstrate teaching collaboration, then you need to plan from the teachers' requirements. This may not need technology; we hope it will benefit learners: but it is organised to benefit teachers.
  

You almost certainly won't get one of these by aiming and planning for another. Our problem was that we had promised in terms of technology use, but actually needed collaboration.

How could we have done it?

Ask "what is AG good for?". This didn't seem to spark much enthusiasm or action. After all, our courses as now running didn't need any new discussion mechanism. That is, as teachers there wasn't any obvious specific need for it or place to use it.

How did we do it?

1. Offline attempts

This did not take off and immediately lead to multiple collaborative teaching instances. However three such cases were done (see next subsection). And furthermore, these offers may have contributed to agreements made at the meeting (subsection 3). I am aware of the following case. I made a general suggestion in an email that teachback between students is a good idea on general pedagogical grounds, and would probably work particularly well across HEIs. Tom Buggy posted suggestions for collaborations he would like to be part of, one of which was such a teachback situation. Before the meeting, I re-read these old emails, and during the meeting as a result suggested teachback to Phil Gray as a worthwhile offer; and when he agreed, suggested to Julian Newman that he phone Tom Buggy (who wasn't present at the meeting) to see if he would like to be the partner on this; and a collaboration on this was agreed.

2. Actual collaborations before the meeting

Two of the cases were of actual student experience of CSCW technologies in courses whose subject matter included CSCW: they were then in part lab. exercises, and to experience CSCW you more or less need collaborators at a remote site. These cases were: using BCSCW software; and using Superjanet video conferencing.

The third case was guest teaching by David Benyon of Napier for Alistair Kilgour of Heriot Watt. This did not really involve any special technology; and it was in a long tradition of occasional guest lecturing.

3. The meeting (Project meeting/workshop of 27 May 1997)

What was actually done was:

1. Presentations, which however do not seem to have been important for the organisation of collaborations. However they will have served as a status report for the project. In particular, the presentation about software basically took the decision to rewrite from scratch, leaving the feeling that decisions could be taken apart from that: specify the software later, rather than trying to decide the software and plan the teaching around it. This may have been important in allowing the technology to be backgrounded.

2. Morning small groups, people from different HEIs carefully mixed in groups, each group had a topic-to-be-taught area. They were supposed to discuss their assigned topic area and how to collaborate.
   No-one seems to have felt their time was wasted, so all the groups were probably useful to the individuals involved. But from the viewpoint of what was productive for the meeting as a whole, the important outcomes were a) the idea of changing the afternoon's agenda to one of proposing and bidding for units; b) 2 ideas on what the units of collaboration should be. [Was it significant that most of this came from the group with Phil and Terry in: the most critical and the most enthusiastic w.r.t. collaborating in teaching: maximised the tension, and got the biggest contribution to the solution.?]
   b1) An ATOM: a 1 week chunk of material to offer. [Although only briefly discussed, this seems to mean about 10 student hours of work, expository resources plus a task for the students, the idea of using an AG-like tool to supplement this, and suggestions for assessment by the author which would however be the responsibility of the local deliverer.]
   b2) Student projects, where one HEI's students do the design, and the other's do the evaluation.

3. Plenary. Making public the different ideas, agreeing on a new agenda for the afternoon.

4. Afternoon groups divided by HEI: 30 mins to come up with a set of offers of ATOMs that could be delivered.

5. Some visiting of other groups to strike initial deals i.e. accepting others' ATOMs.

6. Public listing of
   a) ATOMs on offer
   b) deals already agreed
   c) ATOMs wanted (none of these actually declared at this point, but there were some indications that more of this would have been good).
   d) Then people "voted": declared which ATOMs they would be interested in receiving.

How was it?

1. It felt good to everyone. We'll have to see what actual action outcomes there are, but currently I think we will now proceed to a considerable number of teaching collaborations.

2. What was different or significant? Basically:
   
   • Discussing teaching, and what we could collaborate on.
     
   • Thinking about topics to be taught (or activities for students to put on): both what we could do for others, and what we would like to take from them for our own courses.
     
   • The particular idea of an ATOM
     
   • Being very specific (about units of exchange for organising collaboration)
     
   • The structure of the workshop:
     
     • Having a workshop (not email exchanges), devoted specifically to collaboration (not technology or general project business)
       
     • Small group discussion format
       
     • The game playing metaphor or structure for negotiating collaborations. [This was of diplomatic negotiations between HEIs, trade missions, ....]
       
   • Casting the net wider: inviting participation from other HCI teachers, not just those who are part of MANTCHI.

3. What is and isn't this?
   *It will bring about HEI collaboration over teaching (one project aim).
   *It may involve AG: but this is not yet explored or planned beyond a few phrases.
   *It isn't designed in order to improve learning (though this could be a consequence). I.e. we didn't ask ourselves what we could do that would most improve learning and teaching, though no doubt we did pass suggestions through a personal filter of what would be OK by us on quality grounds.
   *It isn't designed to support learning by overhearing (tertiary courseware?), although this could be a consequence.

4. Weaknesses of how we did it.
   
   • What people (teachers) want, probably not what they most need pedagogically (e.g. Lachlan's stuff on group dynamics)
     
   • What did people mean by vague ATOM titles. (Should have had people explain them in the plenary.)
     
   • My own problem: I still can't bear to think on what part of my course to sacrifice.
     
   • Should have considered and systematically bid for other kinds of entity as well as ATOMs. I.e. ATOMs got almost all the communal attention, although 2 other formats were mentioned and used, and we should have spent more time coming up with other formats. For instance the teachback deal (students from one institution teach a topic to others), and the project design and evaluate swap: these were done and one or two such deals organised, but in general could and should have worked out such types and bid on them more systematically.
     This is also a pity because the ATOM format is regressive in that it is a unit of curriculum: almost an instructivist approach to course redesign. This is in contrast to, say, the teachback format which is about better student activity that could be applied to any topic.

5. Is this a ped-led way of designing project activities? See later section.

6. How does this help the project aims? See later section.

7. Agenda shift. Previous project meetings had tended to focus on technology issues, and not get down to specifics of teaching. The agenda for this meeting was subject-topic (what was to be taught in collaborations) first, then technology (already an important shift). Then the agenda was again changed during the day, so that the focus became a) format (i.e. learning activity) first e.g. ATOMs b) subject topic to be taught c) no time left for other details of the teaching e.g. format d) no time left for discussing technology.

Possible types of cross-HEI collaboration

1. ATOMs: common chunks of course material. About 10 student hours of work, expository resources plus a task for the students, the idea of using an AG-like tool to supplement this, and suggestions for assessment which would however be the responsibility of the local deliverer.

2. Project work student collaboration: one class evaluates the project products of another class. E.g. each designs a thing; and other class evaluates it.

3. Teachback across HEI classes. E.g. 2 week 2 topic chunk. In week 1 each class prepares a topic (different for each class). At changeover, each class teaches the other about their topic (probably one on one student work; could be 1 teaches 3, or 3 teach 1). Then during the rest of week 2 students have to learn topic 2 as best they can, starting from that session. Both topics assessed for credit.
   For receivers: if no gain, they only lost an hour, but potential is to get a useful start. For transmitters: we know it will benefit them a LOT. Just get them to comply. Marks: mainly assess topic knowledge but could also mark the presentations; and use receiver exam marks and grading of presentation too?
   Evaluation: Ask students at end of week 2 how much help giving the teachback presentation was to them; and how much help receiving it was. Look at their exam marks; particularly try to compare benefits of giving and receiving for each student.
   More complex design: each class split in half for topics, so can do within-class comparisons on exams: but this would remove the need for cross-HEI collaboration, while being a better test of teachback alone.

4. As before, use of CSCW technology between HEIs. In effect, a "lab exercise", although the exercise could also attempt real work at the conceptual level at the same time as the content of the exercise.

5. Seminars across HEIs (cf. Netsem). That is, do topics in which students prepare and present by turns; but the presentations are to students at other HEIs as well as their own; and might be by email or video conference.

6. Ditto enhanced by AG. THe idea here is to have an AG of past discussions, from which students will pick a topic for their seminar. In Erica's phrase: this would be a question garden i.e. a collection that shows what good questions are. The garden might be the archive of some email discussion group. My sense is that in many such groups, a few topics recur regularly (besides many others that are dealt with just once), and often this shows which issues remain debateable as well as important to the field. This is on important case of learning by overhearing: eavesdropping (lurking) does actually instruct lurkers about what the issues (not answers) are. In this context, these are the issues that are a good bet for promoting student discussion.

Pedagogy-led vs. technology-led planning

My own theoretical slant

The key difference in deciding what to do is whether you start by asking about how to develop the technology, or you start by asking how to improve teaching. On a smaller scale, the question is where to fit changes into our courses, and how to justify this (to students, to TQA etc.). In medicine, the difference would be between saying "I'm going to try out this new chemical on patients to see what it does" versus asking "What disease is there where we need improvement so much that a new unproven drug is likely to do more good than harm?".

It does NOT follow the route (which early MANTCHI meetings in effect did) of asking "what existing software is available?" then "How can we install it" then "You ought to use it (for the sake of the project, not the students)".

Is this a fair view?

Talking to other project members suggests it does get at an important and central issue, but also suggests important qualifications.

Aversion to educational prescriptive pronouncements

Tacit expressions of pedagogical concern

As things turned out, it would seem that project members show a considerable spread on this issue. Many reacted positively to the bidding process, seeing others' offers often as opportunities to replace topics in their courses they felt least good about.

Spending meeting time discussing technical (software) issues

Urgency

"Social dynamics": how the meeting was organised

Was it really ped-led?

1. Yes: not technology-led but teaching&learning-led.

   However, it was NOT done in the way I argue for in my Niche paper: not by analysing what the biggest need or lack is in a current course, and then looking for something to address that bottleneck. Instead, in our meeting it was to do with a) units we could conceive of making changes around (not too big, related to the curriculum units we tend to think in terms of, made specific by specifying student activities). b) some self-interest: what would reduce our teaching burden and/or be interesting.
   

2. No: the ATOM idea is about units of curriculum i.e. content.
   
3. No: Self-interest: how it would reduce teachers' work, not improve student learning.
   
4. No: not greatest pedagogical need (although that would be one criterion used for accepting others' offers i.e. drop topics we didn't feel we were delivering well).
   
5. Yes: where would the input of collaborators fit in and carry a real part of the course.

How I'd like to spend meetings

What's all this got to do with the project aims?

Problem 1 People begin by feeling that project meetings should do project business; and that software use should go from requirements, to implementation, installation, use. Instead, we should have realised that teaching redesign and collaboration are major activities in their own right, that take an expensive amount of time: needing organisation and planning. They are a major project action in their own right, requiring significant resources.

Problem 2 Our project mainly promised to apply and try out AG software. But the collaborations we discussed were designed to be teaching and be collaborations across HEIs. They were not designed to use AG, nor to use the MANs (although these may well get used as a side effect). So on the face of it, collaboration is not addressing the main aim of the project, so how can we justify a focus on it.

Answer 1 The proposal promised use of AG software across the MANs. This can't be done without collaboration. The mistake was not to recognise that collaboration is a big thing in its own right, and must be addressed as a major activity in and of itself. It was as if we had promised to try out a drug in clinical practice without having made any plans about whether that was ethically acceptable, who the subjects would be, how to get a control group, etc. This planning wasn't done in the project proposal, so it must be done as a project action: even though it wasn't listed as a project objective, milestone, or activity.

Answer 2 If we achieve real and useful collaboration, that will actually be an important deliverable in its own right, and SHEFC will probably be interested. Secondly, we probably will get to use AG as part of it, and if so, then it will be in a sensible and meaningful context. Thirdly, the MANs will get used to some extent: how much is part of what we will discover; and again this will be in a meaningful context where other alternatives have been considered, rather than as a mere demonstration divorced from serious application.

Conclusion on the ped-led issue

• If you want to demonstrate technology use, then you will plan from the technology. (But this has almost always led to indifferent quality of learning.)
  
• If you want to demonstrate improved learning, then you need to plan from the learning needs. (But this may not need technology. See Draper, 1997.
  
• If you want to demonstrate teaching collaboration, then you need to plan from the teachers' requirements. This may not need technology; we hope it will benefit learners: but it is organised to benefit teachers.

What is in fact required to achieve politically and scientifically major results is to show that technology leads to improved learning, in this case using collaboration. This means a much more difficult design aim, of finding a way of simultaneously organising collaboration, technology use, and improved learning outcomes. Neither we nor other projects even attempt this.

Dropping topics to make room for collaboration

Against change

1. In the short run, external TQA, internal academic committees, and giving students proper advance notification of course content all militate against course changes of either content or method (types of student task). Two years advance notice may be necessary for major changes (e.g. adding or dropping a topic like CSCW from the curriculum), at least in principle.

2. HEI teachers often design courses to teach what they are interested in. Dropping this for something else is less attractive.

   In favour of taking help (collaboration)

3. On the other hand, when you have to design a new course, or redesign an old one, you are open to help: in fact probably keen on getting it. This is an opportunity, but typically on a 4 year cycle i.e. once in 4 years.

4. On a smaller scale, a teacher may have to take over part of a course they haven't taught before: the topics may be fairly fixed, but they will be keen to accept material on it from elsewhere.

5. There may be topics in many courses that the teacher feels are least well done or least interesting to them (but taught because they seem necessary to the subject): favourites for dropping or getting them done by outsiders. This means at any time there may be 10% they would be most ready to change.

6. But collaboration might not involve a change of topic, but additional and supplementary activities for students (e.g. teachback). This would favour units of collaboration other than ATOMs or expositions.

My reluctance

Proposed future recipes

Recipe 1

Begin by circulating (1) a written agenda (this recipe), together with (2) some ideas on units of collaboration, and suggest that participants come with proposals.

Plenaries, small groups, and face to face bidding are all essential ingredients.

1. Short introductory plenary talk to warm people up: present the state of the project, and why it is important and urgent to get collaborations organised.
   
2. Divide into small groups to discuss the units of collaboration, mixed across HEIs. Start by discussing the ones suggested here, and decide whether to accept, modify, or reject them. Propose new ones.
   
3. Plenary: pool the results so all know the main ideas on units of collaboration.
   
4. Small groups divided by HEI. Each HEI decides
   a) what topic+unit types to offer;
   b) what they would particularly like to accept if anyone will offer it;
   c) what bits of their current course they will drop in order to make room for receiving others' units;
   d) what the most important problems in their current courses are that they would like to improve. 30 mins should be enough.
   
5. Plenary to share these: write them up on a board, but also have each briefly explained by its author. Get preliminary bids/ expressions of interest on the spot.
   
6. Divide up again somehow by possible collaborations (need an idea how to schedule this as each person will often be part of several potential collaborations). Refine each such bid.
   
7. Back to plenary: start to worry about balance of trade deficits.

Recipe 2 (Wider scope)

1. New deliverers who may not be expert in various necessary topics: they would particularly welcome material from outside.
   
2. Redesign asking what topics would be best for students, as opposed to most interesting or convenient for the teachers: this may throw up things the teachers would welcome help on.
   
3. Redesign courses with collaboration in mind as a basic factor.
   
4. Consider activity type independently of curriculum units: collaboration may be in units like teachback or email seminars that are content-independent. Invent new alternative types.

Possible project top level conclusions

AG is still only an intermediate instrument to help debugging course exposition and delivery. But this could be powerful in letting collaborating academics a) develop only small bits of course (others are doing the rest); b) try it out on several classes per year: faster development; c) software assistance in gathering this feedback to the author is more likely to be justified in this mass approach, and in cross-institution work.

Cost savings (if any) will be in author time: sharing materials development across HEIs. No savings in delivery (each local course manager has to be able to deliver it and examine it).

Separately, student cross-HEI collaboration may have a big value.

The development and dissemination of our method of setting up teaching collaborations across HEIs.

Summary

• Desparation: we had to get the project moving forward, and soon.
  
• The appetite for discussing teaching with other teachers is great and almost entirely unsatisfied at present in HEIs.
  
• Small groups, and the bidding process, and a workshop were probably important for the dynamics to work (as opposed to trying to do it asynchronously by email)
  
• Focussing on units that teachers do or can plan with. The ATOM idea. Making collaboration very specific, rather than an abstract aim.
  
• Focussing on teaching not technology is a crucial part of this. If you want teaching collaboration, focus on the teaching (not the technology, not collaboration in the abstract, not even learning improvements).
  
• Organising collaboration is a major activity in itself. It should be in project plans, with explicitly allocated resources, timetables, and as a deliverable and goal.

References