Last changed 12 Mar 2001 ............... Length about 1,000 words (9,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/complab.html. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [grumps] [this page]

Design of the computing lab warmup study

by Stephen W. Draper

Contents (click to jump to a section)

The software

Phase 1: Huw's collection software will start up whenever a student logs in. It is small (few lines of code), and (subject to various adjustable parameters) will send copies of a wide selection of events including all keystokes to our software and storage. Will start running at week 1 start of summer term.

Phase 2: Small window attached to [P1] for students to signal to the tutoring staff for help. Software elsewhere will catch these signals, and present a queue for tutors and others to view. Will start (be enabled) week 2 of summer term.

Phase 3: Tutor interface. Will allow tutors to define a "milestones" scheme for them to enter how far each student has progressed on the current assignment in terms of a simple scheme, and to present that to tutors on demand, together with other information about that student e.g. how much help they have had from tutors (occasions and duration). Might deliver its IO on PDAs, on web pages, or on the students' machines (tutor leans over and enters stuff).

Phase 4: Further student interface. Will allow students to inspect their logs (records) so far.

Phases 3 and 4 may not be implemented in time for this study.

These phases are in order of time (when turned on), and of priority according to the last Grumps meeting. They are functionally separable implementation jobs. However they are probably NOT in order of value to the educational potential of the project, and may not be so separable from that perspective.

Summary: why do this project?

Here is a sketch of a top-down design for the computing lab startup Grumps project i.e. an attempt to state the aims, techniques we might use to approach them, implications for the implementation plans.

Educational aims

The overall aim is to explore various ways ICT might improve the effectiveness of labs. None are at all certain: we will explore quite a few possibilities.

Top level goals/issues (and hence measures)

  1. Deploy tutors better
  2. Analysis of learners' logs (later) in detail.

1) Tutor efficiency/efficacy
  • Must define "better": gather relevant factors including what tutors say they consider/worry about now; then both measure and support each.

    Techniques to address (1)
    1. Record student actions, especially time on various applications
    2. Record student progress: rely on tutor entering milestones.
    3. Record help episodes (duration, when, when in relation to help request and student actions since -- is it dead time?).
    4. Have a public (mutually seen) queue to allow better allocation by tutor of whom to see, and of waiting time by student.
      This implies, for fuller success, having the queue re-sortable by tutor in various ways to facilitate various scheduling rules; including geographic (go round machines in order but only the machines that actually have my group at them), and "who haven't I seen today?" i.e. by students including those not on queue.
    5. Have a queue entry = help request labelled by topic, so as to allow students and tutors to increase the occasions when several students listen to same interaction.

    2) Log analyses

    HCI aims

    The top level issues are:
    1. The privacy issue. So: is our approach effective in a) managing user fears, b) keeping them aware that logging is in progress.

    2. Queue mutual awareness. Are our techniques effective at this? What does it take to design this, and the many implied issues (undo for losing queue items, how to get both parties separately to rate the entry as it is signed off, ...).

    3. Supporting the priority decisions that tutors actually want to make; and how these shift within one session. E.g. offer the queue as a big table, and have it sortable by any of the 10-20 attributes as support for various possible priority schemes the tutor may be thinking of.

    4. PDA and delivery of tutor information to mobile tutors. Explore PDA vs. student screens, vs. big screen or projection on a wall, ... I.e. the tutors are mobile but in a closed space, and the IO could migrate over fixed devices or have a fixed outlet in a mobile device; and the input needn't use the same solution as the output.

    DIM aims

    I leave it to others to keep an eye on this, BUT see the one item marked ** below, about considering a layered approach to fast-access (as opposed to archived) data.

    Comments

    This section makes remarks about issues bearing on the decision on Thur. about our implementation plan. Items marked * seem most important/difficult.

    *We must set up a data input channel to record help sessions (both start and end times): centrally interesting events. This implies making sure phase 3 happens in time. Without this, we won't really have any data on help interactions.
    This could be done by a) tutor PDAs; b) input window on student machine that tutor leans over and clicks; c) if tutors had location aware-sensors that reported nearest student machine, then could get help start-end from that automatically.

    *We need student text input to the help queue if we are to explore displaying topics on the queue: something tutors sound keen on, and with one of the better hopes for real educational benefit.
    Either extend Huw's thing with a text input box; or have a button click summon another application/window ....

    Currently my vote for the "grading" and cancellation of a completed help session would be selecting one of these: Given up and left the lab, solved it myself, another student solved it for me, tutor came but no use, tutor came - partly useful, tutor came and almost completely resolved it.

    *?Hidden key combo to kill Huw process as an added safety device. (We need to spell out what tools Grumps personnel will have for rescuing any disaster with our software in the lab.)

    Develop spec. of phase 3 (tutor interface) software. Tutor input would be: help start (time), student ID/name/machine, milestone, help end; prompt student to cancel queue entry with a satisfaction rating.

    Phase 4: showing students their records so far. This is an extension of the HCI privacy theme; but also could have educational benefit in promoting "reflection" and self-management of their learning.

    Student input to queue might be both words and a pointer (e.g. type digit to refer to an existing entry) to another entry they see as similar. The queue would then put these side by side and visually linked; but separate so as to allow separate "cancel" ratings and separate physical visits.

    In fact the ideal for tutors is to have a larger queue table, where each entry has a topic, topic ID iff linked to other entries, student name, student machine, amount of help already received (total this year in occasions and durations), time since last help, current progress milestone, times in lab. It would allow them to re-sort the queue by time of request or any of the above attributes (e.g. nearest machine, to minimise walking).

    **I think an issue here is developing the software to allow fast turnaround summaries to be made available. Offline querying only is not really good enough for any of the promising educational benefits. The queue, for instance, may get about one transaction per minute, and require displays to be updated within one or two seconds. Offering a similar service for student records (P4) would be less vital, but highly desirable for gaining trust and interest. Could consider doing this at the level of application switches if not keystrokes. We will also probably need to maintain a fast access table relating student name, ID, machine ID, place on map.

    We should record and publicly display as much data about tutors as about students: part of the equal opportunity aspects of privacy. Even if some tutors aren't very scrupulous about hours worked, they will probably still be better than the student mean. And if the recorded data prompts student complaints about scheduling of help, these comments will be interesting for both staff and us.

    If we can make acceptable the public display of every student's milestone this will make many things easier for us.

    Web site logical path: [www.psy.gla.ac.uk] [~steve] [grumps] [this page]
    [Top of this page]