Last changed 16 May 2017 ............... Length about 5,000 words (39,000 bytes).
(Document started on 6 Aug 2010.) This is a WWW document maintained by Steve Draper, installed at You may copy it. How to refer to it.

Web site logical path: [] [~steve] [localed] [this page]

Current L&T innovations to adopt

By Steve Draper,   Department of Psychology,   University of Glasgow.

This page lists learning and teaching innovations that I think we should probably adopt immediately. Although I'm working on helping my own department to introduce many of these, I am eager to encourage and assist others at Glasgow University (GU) to do so too; and indeed elsewhere as well. In some cases I have an assistant for this as well, so you may be able to get more substantial help than just hurried advice. In any case, if you are interested then feel free to get in touch, whether or not I am the "main person" for the idea or software.

Contents (click to jump to a section)

  1. Advancing academic writing (AAW): web resources for active student development
  2. Maths / numeracy: web resources for active student development
  3. Feedback Calendars
  4. PeerWise: software to administer student authoring of MCQs in large classes.
  5. Aropä: software to administer reciprocal peer critiquing in large classes.
  6. Small group RPC (Reciprocal Peer Critiquing)
  7. Student-generated PDP
  8. Choosing a third subject

Advancing academic writing (AAW): web resources for active student development

(Main/ other web page on this:  
Page for staff (adopters)   the moodle resources   Katie Grant   my old intro   funding application )

What is this? Advancing Academic Writing (AAW) is a substantial interactive moodle-based website designed to help students improve their writing skills. The site comprises information and extensive exercises, with corresponding paper feedback/action sheets for use by markers. The marked up feedback/action sheets link particular bits of student work to particular writing skills topics and exercises. For maximum relevance to students, the information, exercises and feedback/action sheets are subject specific. Each subject has its own moodle portal and the materials have been developed from extensive reading of real student essays.

Why is it a good idea? (Theory and evidence) Staff are often dismayed by the inability of students to write with focus and clarity. Students, aware of their own shortcomings, want to improve their writing skills both to "unlock better grades" (as one student described it) and to enhance their employment prospects after graduation. AAW provides an opportunity for staff not only to make diagnostic comments easily, but also to point to a site that can help sort out problems currently acknowledged but not addressed. For students, AAW provides an ongoing, specifically tailored resource. Feedback from students and staff so far is positive.

Useful points:

Typical learning designs using the idea All tutors who are marking student work tick boxes on an additional writing skills feedback sheet, perhaps embedded into existing feedback sheets, whether digital or paper. Students take the sheet, go to the website, click into their subject, then click into the categories ticked on their sheet to find the explanations and exercises they need.

Papers on this software

  • "Writing for results" conference paper (here)   Abstract   Slides   Handout
  • (We expect to write more.)

    Pointers to the software The moodle resources (anyone can login as a guest)

    Cost / risk / sustainability Following a successful pilot year, there is now a one-off charge to departments (or other disciplinary teaching units) for subject site development. There is no charge for upkeep of the site and, if they wish to, disciplines can update their own resource with moderate HTML/moodle skills.

    How much work first time, to start this up?: The first step for the adopter is to get their unit to decide to implement AAW and to pay the project. Once this is done, the request is an email or a phone call (see below for contact details). About 15 pieces of appropriate student work (essays/reports) should be made available for collection. There will be discussion as to styles (preferred referencing style, preferred capitalisation etc.). The content of the feedback/action sheet will be agreed with the subject staff involved. The site will be set up and populated with information and exercises. Staff will need to familiarise themselves with the materials before using the feedback/action sheets.
    How much extra staff marking time, assuming familiarity? A few seconds for each student script.
    How much student time per delivery? None before handing work in, but perhaps several hours (by themselves) if they follow up the pointers. In a pilot trial, students often spent more time than they expected. Some of this time was voluntary, i.e. once they had done the work indicated by ticks on the feedback/action sheet, they saw other categories on the site they felt of use to them.

    Existing users at GU:
    Already used it on classes English literature, History, Computing Science, Arts Academic Writing Programme
    Will use it on classes in Sem1 2010 Earth sciences, Engineering, Philosophy, Psychology
    Negotiating about having a site built by the project Classics, Management, Biological sciences

    How can I action this? Contact Katie Grant. You will need to:

  • Persuade your department or unit to adopt it, and to pay for it (pricing structure).
  • Make samples of problematic student writing available for collection.
  • Organise tutor training: this will be quick and we will help, but becoming familiar with the feedback/action sheets and the website will speed up the benefits.

    Funding: LTDF 2009-10 £30,000;   LTDF 2010-11 £25,000 plus payments from departments.

    Main person behind it: Katie Grant.

    Maths / numeracy: web resources for active student development

    Main/ other web page on this:  
    my old intro   my newer index page

    Feedback Calendars

    (Main/ other web page on this:  
    Main web page   An example calendar   funding application )

    What is this? These are tables to be included in course documentation showing all the feedback students get, with dates, types, etc.

    Why is it a good idea? (Theory and evidence) It simply draws together disparate bits of information on this one aspect of a course into one place, thus encouraging both staff and students to take an overview, to plan their actions accordingly, and to understand how their actions fit in with others'. The hope is, that this will gradually improve both students' appreciation and use of feedback, and staff's appreciation of how feedback can or should fit into their courses, and perhaps eventually prompt improvement in the timeliness and variety of feedback.

    NSS point.

    Typical learning designs using the idea These are not directly associated with new learning designs in the sense of joint activities such as lectures, labs, compulsory assignments. The concrete plan is simply to create the calendar, probably as a table, and distribute it for reference to staff and students. The actual possible resulting changed actions are not timetabled ones. For staff it may affect course design (modifications) whenever that is done. For students it may influence the attention, and hence perhaps the time, they give to feedback and possible ways of acting on it. There is no learning design directly associated with introducing the calendars. They may prompt redesigns of (or more likely, adjustments to) existing activities with feedback.

    Papers on this None yet. This is a new idea, that as far as I know, has never been done before (August 2010).

    Pointers to the software The main web page both shows examples and gives downloadable WORD templates for creating your own. See the main web page.

    Cost / risk / sustainability Very fast. Benefits, if any, later. Sustainable.
    How much work first time, to start this up?: A few hours assembling and revising draft versions of the calendar (and looking up the scattered information for this).
    How much staff time per delivery (once running well): none
    How much student time per delivery: None

    Existing users: Psychology level 1, Psychology level 2, Psychology level 3, (Management level 1)

    How can I action this? Contact Steve Draper. You could do it yourself, but it may well feel less effort and you will be less likely to let it slide, if I (and perhaps an assistant) help you fill in the format, suggest entries, do the editing, etc. Furthermore, the benefit is in being prompted to think about feedback provision, prompted by a new format. Discussing it with someone else is generally less effort and more stimulating than just filling it in alone, and so will probably be interesting for you, and certainly for me. These ideas are new, and there is no published framework for thinking about feedback provision overall. So for me, each new client prompts new ideas ...

    Funding: LTDF 2010-11 £5,000

    Main person behind it: Steve Draper

    PeerWise: software to administer student authoring of MCQs in large classes.

     See folders on my office desk 
    (Main/ other web page on this:  
    PeerWise website   Paul Denny's web page   my old intro )

    What is this? The idea is to get students to create test MCQs (Multiple Choice Questions) for other students to use. Writing a decent question turns out to require much deeper learning than just answering one. It works best in large classes; and there is now available software to manage the administration so completely that there is little additional work for staff.

    Why is it a good idea? (Theory and evidence) For a general argument that students writing MCQs can be a powerful lever for their learning, based on extensive lab studies and classroom cases, see this paper on "catalytic assessment".

    Typical learning designs using the idea [1-2 cases] A class of 300. Perhaps 20 minutes of a lecture or tutorial on what makes a good question. Credit bearing exercise where each student must create 2 questions, and answer 10 and rate them. Best done by half-term; there will be a second rush of use of the created question bank at revision time.

    Papers on this software
    Perhaps start with this one:
    Denny, P., Hamer, J., Luxton-Reilly, A., and Purchase, H. (2008) "PeerWise" In Proceedings of the 8th international Conference on Computing Education Research (Koli, Finland, November 13 - 16, 2008) (ACM: New York, NY) pp.109-112

    Or a youtube video:

    And a 2nd paper to look at might then be:
    Denny, P., Hanks, B., and Simon, B. (2010) "Peerwise: replication study of a student-collaborative self-testing web service in a U.S. setting" In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (Milwaukee, Wisconsin, USA, March 10 - 13, 2010) SIGCSE '10. (ACM: New York, NY) pp.421-425

    More papers (see also Paul Denny's web page):

  • Paul Denny, Andrew Luxton-Reilly, John Hamer and Helen Purchase. "Coverage of course topics in a student generated MCQ repository" In ITiCSE'09: Proceedings of the 14th Annual SIGCSE Conference on Innovation and Technology in Computer Science, pages (to appear), Paris, France, July 2009. ACM.
  • John Hamer, Paul Denny, Andrew Luxton-Reilly, John Hosking, & Beryl Plimmer "Aropä and PeerWise: Supporting student contributed pedagogy in large classes" EDUCAUSE Australasia 2009, pages (to appear), Perth, WA, Australia, May 2009
  • Paul Denny, Andrew Luxton-Reilly, and Beth Simon "Quality of student contributed questions using PeerWise" In M. Hamilton and T. Clear, editors, Eleventh Australasian Computing Education Conference (ACE 2009), volume 95 of CRPIT, pages (to appear), Wellington, New Zealand, 2009. ACS.
  • Paul Denny, Andrew Luxton-Reilly, and John Hamer "Tools that support contribution-based pedagogies" In Proceedings of the ACM-IFIP IEEIII 2008 Informatics Education Europe III Conference, page (to appear), Venice, Italy, December 2008
  • Paul Denny, John Hamer, Andrew Luxton-Reilly, and Helen Purchase "PeerWise: Students sharing their multiple choice questions" In Fourth International Computing Education Research Workshop (ICER 2008), pages 51-58, Sydney, Australia, September 2008.
  • Paul Denny, Andrew Luxton-Reilly, and John Hamer "The PeerWise system of student contributed assessment questions" In Simon and M. Hamilton, editors, Tenth Australasian Computing Education Conference (ACE 2008), volume 78 of CRPIT, pages 69-74, Wollongong, NSW, Australia, 2008. ACS.
  • Paul Denny, Andrew Luxton-Reilly, and John Hamer. "Student use of the PeerWise system" In ITiCSE'08: Proceedings of the 13th Annual SIGCSE Conference on Innovation and Technology in Computer Science, pp.73-77, Madrid, Spain, 2008. ACM.

    Pointers to the software PeerWise website The software is currently offered as a free web service. Email them to create an account, and off you go.

    Cost / risk / sustainability Zero cost; no reports of serious service crashes so far. But this depends on an academic site in New Zealand. In the longer term, other arrangements will presumably become important; but for now it seems a good bet; and many institutions are using it (45 currently).
    How much work first time, to start this up?: A few minutes organising access to the software. Some time to familiarise yourself with the software. Finding a place for it in your course schedule.
    How much staff time per delivery (once running well): Very little: the software does it all.
    How much student time per delivery: Perhaps an hour per question written; and a minute per question answered; plus any extra time they spend reviewing and discussing the questions with other students.

    Existing users: Many institutions are using it (45 currently), according the PeerWise front page. Paul Denny gave a 2 hour workshop on it at GU in July 2010. I expect several GU courses to start using it next semester.

    How can I action this? Email PeerWise for a login (very quick); create a repository of your own, and off you go. There is documenation, but it seems pretty easy to use.

    Main person behind it: Paul Denny

    Aropä: software to administer reciprocal peer critiquing in large classes.

     See folders on office desk 
    (Main/ other web page on this:  
    the resources   my old intro )

    What is this? Students produce and communicate critiques of each others' work, usually anonymously via the software. Do NOT confound this with peer assessment (students giving marks to fellow students that count towards course grades).

    Why is it a good idea? (Theory and evidence) It exercises their judgement, not just as writers, but as readers and critics. The key bottleneck in feedback is in getting students to understand the key criteria e.g. critical thinking. RPC can be a major help here, by developing their understanding of these criteria by exercising them in varied ways. The critiques produced are also useful to learners, and greatly increase the amount of feedback they get.

    In a large class, doing the clerical administration of this (copying and distributing large numbers of student essays; collecting the critiques and delivering those, ...) is a major task: but free software exists to do it.

    Typical learning designs using the idea [0-2 cases] On a piece of coursework, students are required having submitted their own work, to produce comments on 2 or 3 other students' work. They might then be required to "review the reviews": comment on an critique the reviews of their own work.

    Papers on this software

  • John Hamer, Catherine Kell & Fiona Spence (2007) "Peer assessment using Aropä" A report on a research study into three courses that have used Aropä. (ACE2007).
  • John Hamer, Helen Purchase, Paul Denny, & Andrew Luxton-Reilly (2009) "Quality of peer assessment in CS1" ICER'09 (ACM)
  • Also see John Hamer's page for pointers to several papers, slide-sets, and a video.

    Pointers to the software There are a number of bits of software that support this to some extent, although I'm most impressed by Aropä so far.

  • Aropä
  • There is a module written for Moodle (but only in Spanish)
  • The Moodle module "workshops" aim at supporting it, but isn't that good. (See also
  • You could use (free) software meant for administering refereeing papers for conferences. (Quintin Cutts has done this with "OpenConf"
  • TeCTRA: Raban & Litchfield at University of technology Sydney, papers in Ascilite -
  • WebPA - Loughborough University
  • or papers on peer assessment rather than helpful formative peer critiquing

    Cost / risk / sustainability No charge. John Hamer is in Glasgow currently, and is creating and maintaining an installation of the software here at GU.
    How much work first time, to start this up?: A few hours organising access to the software, and writing the "rubric" guiding the student reviewers for your subject and purposes. Finding a place for it in your course schedule.
    How much staff time per delivery (once running well): Very little: the software does it all.
    How much student time per delivery: Several hours work per exercise. It depends what you require of them e.g. how many reviews, how long the work is that they have to read and critique.

    Existing users: An impressive variety of disciplines from arts to science have used it at Auckland including: Academic Practice, Chemical and Materials Engineering, Civil Engineering, Commercial Law, Computer Science, Electrical Engineering, English, Marine Biology, Medical Science, Pharmacology, Population Health, Software Engineering.
    It will be used in computing science at Glasgow this autumn.

    How can I action this?

  • Contact Helen or John; get access to the service.
  • Find a place for it in your course schedule.
  • Create the "rubric" specifying the judgement criteria and hence the kind of comments students should give: i.e. design your version of the exercise

    Funding: HEA-ICS 2010-11 £3,500

    Main persons behind it: John Hamer   Helen Purchase

    Small group RCP (Reciprocal Peer Critiquing)

    (Main/ other web page on this:  
    Main web page   my old intro )

    What is this? Students produce and communicate critiques of each others' work, written but delivered face to face in a meeting chaired by their tutor.

    Why is it a good idea? (Theory and evidence) The general reasons are as above for mass, anonymous RPC with Aröpa: developing their grasp of core disciplinary criteria by exercising them as readers and critics as well as authors. However delivering the comments the first time with a tutor as chair has the effect of making them have confidence (feel protected), take the exercise seriously (no frivolous remarks, no failure to read the work carefully or they will look foolish), and generally set the tone for the exercise. Having survived this once, and found it useful, they then frequently organise it themselves without feeling any need for a staff presence.

    It seems also to accelerate group bonding and cohesion: they have done something that is both academically serious, and personal; and have benefitted from each others' contributions. An experience of the academic (not merely social) benefits of peer interaction and group work.

    Typical learning designs using the idea [1-2 cases] At the start of a semester, and in the first meeting in a new tutorial group, organise the first set of RPC. Then after the comments are delivered (e.g. in the second meeting), ask each student whether they found it useful, and in what ways. Then say: if you found that useful, how about committing to finishing a complete draft of the next piece of work a week early; and repeating the exercise, finally submitting the work to staff after improving it in the light of peer comments. (This of course depends on them not all writing the identical essay topic.)

    Papers on this method

  • Morrow,M.I. (2006) "An Application of Peer Feedback to Undergraduates' Writing of Critical Literature Reviews" Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education" vol.1 no.2 pp.61-72
  • A talk on it (abstract, slides, ...)
  • Another talk

    Pointers to the software No software for this (sample word documents for copying are available).

    Cost / risk / sustainability No risk or sustainability issues. The cost is making time for it in students' schedules, and getting the tutors prepared to do it.
    How much work for each tutor first time, to start this up?: No prep work, except rehearsing in your head how you will present it to your students.
    How much staff time per delivery (once running well): 10 mins announcing what they have to do; 10 mins copying and distributing work among the students; A session (30-50 mins) that you chair when the feedback /critiques are delivered.
    How much student time per delivery: The above time with you, plus an hour per (large) essay that they review.

    Existing users: Several tutors in my department have used this.

    How can I action this? Help yourself to my web descriptions/resources on this; feel free to ask me.

    Main person behind it: Steve Draper

    Student-generated PDP

    (Main/ other web page on this:  
    Outline idea   Nick Bowskill's PhD elaborations )

    What is this? The intervention consists of sessions (at the start of a year) in which student concerns about a course or programme as a whole are elicited, shared, discussed; and possible solutions too are discussed, suggested by student mentors who have completed the course.

    Why is it a good idea? (Theory and evidence) It is thus a student-generated approach to PDP (Personal Development Planning), which in turn is an important aspect of induction and transition. More technically, it uses (or stimulates) reflection and mentoring to do this.

    * It prompts reflection on being a student on this course. From this viewpoint, the benefit is for individual students, but applied in a mass (cheap) intervention.

    * It produces shared information about the class and their concerns as a whole, so that each individual can locate their own experience and fears in relation to that. This may be a relatively deep way of promoting "social integration" (Tinto's term) by making visible not just things they have in common, but a representation of the amount and kind of diversity. Thus for each issue, everyone is now aware of its presence in the class, and also knows that everyone else is aware of it rather than assuming everyone else is like themselves. Currently, this is directly about "concerns in doing the course" which is often about finances, amount of work, and so on. But it could also raise issues like "worried about being the only male in the class". And in future one could imagine a little shaping might extend this to other issues.

    * By being open to observation by heads of year / programme or course team leaders, it gives them valuable feedback on what the actual student problems are; and/or allows them to show the students more convincingly what they are. (While many of the issues will already be known to staff — so the value there is that having students come up with them is much more convincing to them as in the first point above — there will often also be issues new to staff, so this is a source of learner staff feedback.)

    Typical learning designs using the idea [1-2 cases]

    Papers on this approach See here.

    Hardware / software This uses an EVS (electronic / classroom voting system). It's unlikely you would buy one just for this. For very small classes, you could use a show of hands for voting. For small classes (e.g. 30), an EVS to get votes on what all students think about each short-listed "concern" works well. For large classes (e.g. 300), we used an EVS (e.g. WordWall, Promethian) that supports free-text entry, so that we could get the named concern from each group collected very quickly, to make up the short list that was then voted on.

    Cost / risk / sustainability
    How much work first time, to start this up?: scheduling it is the big thing; getting the EVS equipment booked; ...
    How much staff time per delivery (once running well): 2 half days of sessions per class per year.
    How much student time per delivery: ditto

    Existing users: Nick Bowskill has run it for a number of classes at a number of institutions. At GU, in Sept. 2009 Quintin Cutts did this for the FIMS induction; and this year Steve Draper will do it again.

    How can I action this? For non-GU people: contact Nick Bowskill. For GU people: either contact Nick Bowskill, or ask Steve Draper. I am interested in using this as a practical technique that could be seen as one systematic approach to PDP, and used once a year on all classes. Subject to my time and energy, I could deliver it; and I have access to a set of WordWall equipment.

    Main persons behind it: Original idea: Nick Bowskill.   GU first application: Quintin Cutts.   GU currently: Steve Draper.

    Choosing a third subject

    (Main/ other web page on this:  
    The project page, reports etc.   funding application )

    What is this? What is "3rd subject". This is not an innovation to adopt, but a current small project looking into how GU students choose their 3rd subjects in first year; and how better to support that. We also hope to prototype an information resource (web page? software program?) to assist students and advisors to explore and make the choices. The main output will be reports, which will be made available on a new web page.

    Innovations?: After this project we might recommend sending to accepted applicants, more effective material that would lead to most of them coming to their first Advisor meeting with a provisional choice, (instead of most not).

    Funding: LTDF 2010-11 £14,200

    Main persons behind it: Lorna Love and Steve Draper.

    Web site logical path: [] [~steve] [localed] [this page]
    [Top of this page]