Last changed
24 Apr 2001 ............... Length about 3,000 words (20,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/grumps/lss.html.
You may copy it. How to refer to it.
Web site logical path:
[www.psy.gla.ac.uk]
[~steve]
[grumps]
[this page]
by
Stephen W. Draper
These are my views of the general design rationale and approach to this
software (LSS: lab support software). The current implementation will differ
from this because of insufficient time to do everything, because reality
discovered by the implementors will have undermined or changed some things, and
because other project members will have improved the real design from this
somewhat personal and untested view.
This memo discusses the design of a suite of software for supporting
groups in computing science (and other?) labs: typically, a set of about 20
students, a tutor and a demonstrator. These groups meet at scheduled lab
times, in one place.
The idea arose in the context of a project about recording user input actions
in a computing lab, and asking firstly what are the most immediately useful
deductions that might be made from those records (probably, simple presence
indicated by a login event, and whereabouts in the large lab room, deducible
from which machine in the lab that login was recorded on), and secondly what
are the educational benefits that applying technology could possibly
create here.
This is an extended list of possible educational benefits: it is most
unlikely that all will be achieved in the first version of the software.
- A better managed help queue. A major feature of these labs is requests for
help from students to staff, signalled by students putting their hands up.
Staff wonder whether they are allocating time to students in the best way.
Having a list of students requesting help with as much information as possible
could improve this decision making.
- Save students the fatigue of holding their hands up by offering an electronic
alternative way of signalling, given that many minutes may elapse from first
deciding to ask to receiving a visit.
- If students indicate, by a small text description, what they want help about,
tutors could review the list and spot topics many students want to know about
and give one talk rather than many repetitive consultations.
- Ditto if students review the list, and seeing that another student has
requested a topic prompts them to add their name to it, even if they could not
have articulated the question themselves. Both these are fundamental
advantages of group (as opposed to individual) teaching, though they depend
upon establishing a social tone in the group where students are comfortable
with asking questions in front of staff and in front of other students.
- Facilitate students helping each other. Problems with plagiarism and
assessment may prevent progress on this in the short term. However there is
strong reason to believe that teaching something to someone else is the most
powerful of all learning methods i.e. even if we didn't want to save staff
time, we should be using "teachback" exercises. In the lab, this is a natural
approach. It could be facilitated by having a common help (information)
request queue that students as well as staff could service, rewarded by showing
publicly who satisfies requests, and suggesting and displaying norms about how
much of their time students should spend helping others (neither too much nor
too little).
- Supporting basic staff administration: software could keep automatic records
of attendance, and of staff-student interactions. Thus tutors could check when
they last spoke to or saw each student in the group, and whether this was
because the student failed to attend (perhaps through illness). Even more
basically, summoning up a student's name, photograph, and login name could be
helpful. Equally, letting students see those particulars for the staff on duty
might help interaction. These things are even more likely to be useful in
"drop in labs", where staff and students are even less likely to know each
other by sight and by name; and if one member of staff substitutes for another
e.g. due to illness. Though very low level, such support both saves staff time
in admin., promotes better face to face interaction, and allows staff to manage
their allocation of attention to students better.
- Student progress milestones: where lab work is broken up into small
exercises, software could be used to keep track of each student's progress.
When a staff member sees a work item has been completed, they "tick it off"
through software. This allows staff (e.g. tutor and demonstrator) to
coordinate this without trouble, and could also be used for help priorities
(e.g. visit the students who are furthest behind first).
Even where the work was not designed with this in mind, it would be easy for a
single tutor to define a progress scale privately (e.g. a scale of 0 to 5 for
progress on a large project), and update each student's rating whenever they
see them. As long as the scale is not used for formal assessment, some
informality and inaccuracy is acceptable. Nor is it necessary to conceal it
from the students e.g. "This is my estimate of how far along you are: it's not
a mark. If you think I've got it wrong, surprise me." Ad hoc scales and
definitions for each of their scale points could be set up by a single tutor, a
tutor and demonstrator together, or suggested along with the exercise for the
whole class.
The main information held about the students in a lab group might
include the following. Any or all of it could be used by tutors in deciding
whom to talk to next, and their scheduling policy will normally change during a
single session. Tutors and demonstrators commonly have different scheduling
policies (e.g. the tutor might begin by visiting every student in turn, ordered
by spatial proximity, while the demonstrator tries to answer requests in the
order they are signalled).
- Student name, photo, login
- Machine (IP number, location in lab)
- Whether logged in, and for how long (or time first logged in this session)
- Whether they want help (flag)
- Time since help requested
- Description of help topic requested (free text)
- Link to other students' requests (to indicate identity or relatedness of
topic): could be done as an index number referring to another student or
request.
- Number of requests made. Mean number of requests per lab session.
- Time since previous help dealt with.
- Time since last seen by tutor (both within this session, and last day seen)
- Accumulated time spent with this student: within this session, this year.
- Milestone (if defined, supported, and used by this tutor).
In the interests of symmetry, it is probably a good idea to post
information on the staff too: names, logins or emails, photos, number or speed
of dealing with requests (allows students to estimate wait time) etc. Whether
staff choose to allow this information to be seen by their peers will depend on
the feel of that other group (of staff): obviously it could be useful in
comparing experiences, but won't happen without a friendly and supportive
atmosphere.
Such software is in one way comparable to software supporting any
telephone sales or banking: it is meant to enhance an essentially human-human
interaction by providing instant information from a database to one or both
parties about one or both parties, thus reducing reliance on memory. It is
also in another way like ATMs for banks, in that the software is in direct
competition to a human alternative: if this software is in any way
unsatisfactory, the humans can just ignore it and bypass it with direct face
to face interaction. Thus it is safe (completely non-critical), but by the
same token it a faces much stiffer standards than does software with an
effective monopoly of function (e.g. airline reservation software, or word
processing), where users have no real alternative to using it. Thirdly,
however, this software is not a vast and general internet application but on
the contrary is for supporting a small face to face group. Many heuristics
and first reactions about privacy are therefore totally inappropriate here:
it is foolish to discuss hiding student photographs unless the group
convention was to wear veils, or hiding student or tutor login records, when
this simply reflects attendance at a meeting where of course they are mutually
visible, or hiding either requests for help or their satisfaction, when this
is otherwise done by raising hands in full view of the group and a visible
visit by a staff member to that individual. On the other hand any release of
the data to those outside the group should be carefully considered.
Because this is about supporting a face to face group, what happens will not be
controlled by the software. It will be controlled by the group, and
particularly its leader, the tutor. Simple verbal announcements may be made at
any time, allowing procedural changes, often several in the course of a single
two hour session. A tutor might begin by announcing he is going to come round
all students in turn: this may remove much of the motivation for signalling for
help by any method, since that will now probably have no effect on whether or
when the tutor will arrive at that student, and it will mean students only type
in information on what they want to ask about if they value this as a memory
aid to themselves. Having finished that round, he might call out for a show of
hands from those who now have further requests to get a quick count (faster
than asking them to do it via software), but then tell all those to indicate
their request by software along with a description of the topic they want to
ask about. He may or may not announce the policy for selecting students (e.g.
"put in all your requests, but I'm going to concentrate first on those still
unsure about designing the IO module", or "I'll give priority to those whom
I've spent the least time with so far"). Thus designing the software is
designing a relatively passive complement to interactions determined actively
and dynamically by human actions, some of which are not even mediated, much
less determined, by the software.
The main pieces of software are a small database, Coldfusion which
generates web pages from the database, and a set of scripts specifying this.
Thus the data are visible via web pages (one set for students, and an
additional separate set for staff), and user inputs are also from forms on web
pages. Some of the data are semi-permanent (e.g. which students are in which
group), some are highly volatile (a student has just requested help), and some
are in between (e.g. accumulated records of student logins and help
episodes).
Access can be restricted by machine (IP number) e.g. students can only access
this from the lab; and by password: accessing these web pages requires a login
name and password.
Some of the configuration could be made controllable by the staff on the spot
e.g. when to declare a lab session ended (and clear all requests, etc.), what
fields to reveal to or conceal from students. A further mechanism is needed
to correct any faulty data speedily (e.g. spelling of a student's name,
assignment of students to groups).
The general approach to the user interface design is inspired by Sorgaard
(see also
here and
here).
He pointed out that train reservation systems would be much
better for users if they were designed around a display of the seat layout for
a train that both passenger and clerk could point at. Even a fairly schematic
display will show many relationships, any of which might be of interest to a
few passengers: how near a seat is to the door, the toilet, the restaurant
car, the windows, etc. As with maps, displaying N items also displays N^2
relationships "for free"; and furthermore, the idea of a layout leads to
displaying items you might otherwise not such as the windows, toilets, etc.
This is in contrast to the more familiar alternatives where passengers are only
allowed to refer to two or three attributes (forward/backwards, window/aisle).
The deeper lesson here is that some of the best designs do NOT come from a
process of explicitly identifying user requirements and then laboriously
implementing a feature for each of these. On the one hand, the number of user
requirements (the different kinds of preference passengers form, the ways
tutors make decisions about which student to talk to next) is too great to
identify in advance or even at all; on the other hand, some designs support
more of these than is worth ennumerating. This, by implication, rejects not
only top down design, and the waterfall model of software development, but
undermines the much more basic idea of needing to know the detailed
requirements in order to produce the design, or even of knowing the functions
which the combined human-software system is in fact executing.
In this case, the centre is not a true 2D spatial display, but there is some
analogy to the design approach nevertheless. The idea here is to have a table
with all the student attributes it is convenient to collect, or that might be
important. The display should simply be of all these attributes, but sorted
by any one of them the tutor chooses. The lab "map" display simply functions
here as sort order for proximity (so tutors can select the next nearest
student). Each order corresponds to a different scheduling criterion by the
tutor. Each view of the data should have a direct link to all the other
views, so tutors can switch between them very fast. In future we may have to
consider allowing sorting by more than one attribute at a time. However the
present simple design will probably support much more complex scheduling rules
than we explicitly understand (yet) as designers e.g. a tutor reviews the list
to see how many help requests there are: if few then just see them in
arbitrary order, if many, then check none are more than 30 minutes old,
otherwise do them in order of who is furthest behind in the work. Good
displays can leave complex procedures and relationships implicit in the mind
of the user without any apparent penalty in effort or performance. We may not
understand this, but we know from examples (existence proofs) that this is
true.
Under this approach, any new attribute that seems to be useful can be added to
the table, leaving the general design unchanged, and without any research about
whether enough users care enough about it to justify adding it. The first main
problem we are likely to have to address is the tables becoming too large for
convenient display. That may require something like a facility for an
individual user to blank out some attributes (table columns).
Students can signal for help by a button click on their web page.
Tutors can see these signals in the lists (and map) of their group. They also
see how long a request has been pending, and information on past requests by
that student. That is potentially they can choose who to see next not only on
the basis of who wants it or has been waiting longest, but who has had the
least or most help in the past, on in this lab session.
Students can cancel this signal at any time.
Students record how helpful a visit was, or whether something else solved the
problem. [At the moment, student selects an option with radio buttons and
separately clicks "End request": why not get rid of separate button, and have
selection of a radio button do it?]
Currently, students may have at most one request signalled. This is probably
bad because:
a) In practice students frequently have more than one topic in mind. On the
other hand, they can show this in the arbitrary "topic content" text.
b) It would be good to have students indicate they want to ask about the same
thing as another student.
Students can (optionally) type in a few keywords to indicate the content of
the help they want.
This has been extended to a memo facility, private to that student. They can
store any number of these memo items; can move text between their memos and
their one current help request; can delete them; can choose to have them
visible to the tutor or not.
The content of their one help request only can be seen by other students in
their group only during official lab hours only. The last of these 3
restrictions reflects plagiarism concerns (not wanting to facilitate students
giving each other help), and will probably be relaxed next academic year. On
the other hand, all items ("memos") the student wishes can be visible to the
tutors at all times e.g. in advance of the lab times.
Desirable pedagogic aims here are:
- Allow tutors to spot topics several students want and to arrange to
spake to them all at once, without repetition.
- Allow students to spot a topic another student has articulated, and
recognise and signal that they want to hear it too, even though they weren't
able to identify or articulate that themselves.
- Encourage students to help each other: explaining something to others
is a very powerful learning activity.
The current implementation does support (1). When a change in assessment
approach allows (3) to be pursued, then (2) and (3) could be better supported
by having students maintain multiple topics (as in the current memo facility),
for them all to be visible by all other students, and for a student to be able
to signal a pointer to another student's topic. This will probably mean a new
display page where all topics are listed (rather than all students).
We would like to record (and display to tutors and students) a record of
consultations.
The existence of a completed consultation is signalled when a student
terminates a help request, and records how helpful it was. However if the
visit by the tutor was not requested, a different mechanism is needed.
Currently "last seen at" buttons do this. However the motivation for using
them is less clear; and conversely, they may be pressed at inappropriate
times and it is not clear how easy it will be to distinguish these uses.
End times of consultations are implicitly recorded by the above mechanisms.
Start times are only recorded directly (as opposed to inferred as being the
end time of the last consultation) if and when tutors explicitly click on a
student record. Perhaps a more explicit button would be better, although this
adds to required tutor actions.
Probably a lot more design work is needed here, seeking a solution that will
record consultations and their durations accurately, with as little work
clicking special buttons as possible, robust against user errors (mistaken
button presses), and against misuse by students.
The above, organised by function / intention, can be re-listed in terms of
commands from users to the LSS.
- Help request
- Help content (optional text typed in)
- Memo create
- Memo delete
- Memo reveal/hide to tutor
- Help terminate and grade
- Last seen button: record end of consultation on student page
- Last seen button: record end of consultation on tutor page
- Implicit consultation start: opening a student record by tutor
The software support in no way changes the fact that this is a dynamic
face to face group, with rules that can, entirely appropriately, be changed
frequently. Simply announce from time to time what they are. For instance:
"I'm going to try to come round to each of you in turn now, but the
demonstrator will try to answer urgent queries meanwhile if you signal via the
software. However if you want to remind yourself and me of a topic you want to
ask me about, feel free to put that in the help queue too, but add the word
'tutor' so the demonstrator doesn't waste his time on it." Or "I see there's a
big queue right now, so it may be some time before I see all of you. I'm going
to give priority to those I haven't spent much time with yet, so if you have
already asked a lot of questions you may have to wait longer."
At different times tutors wonder about a number of different issues in
allocating their time: after all, the most vocal students aren't necessarily
the ones really requiring the most attention. The displays let you view all
the available attributes, and to sort the list by any one of them. Whether you
tell the students the order you are going to use is up to you.
The interface is a web page. On accessing it you will be asked for your
(Unix) password: this is what gives you access, and prevents students from
seeing the staff version of the lists.
The map view and list views are linked by buttons. There is a help button for
further explanation of the displays (e.g. map legend).
Ideally (this may not be true in all versions of the software) there will be a
simple action (button press) to perform at the start and another at the end of
each student consultation, which among other things help keep track of how
longs is spent with each student. In addition, please prompt the student at
the end of each consultation to signal this and so cancel any help request on
their window.
There may also be some tutor-only controls that affect the configuration for
your group session e.g. declaring a lab session over, clearing help requests en
bloc (e.g. at the start, or after giving a talk), controlling what fields are
displayed to your students.
The interface is a web page. On accessing it you will be asked for your
(Unix) password: this is what gives you access, and prevents others from
seeing your private memo entries.
The map view and list views are linked by buttons. There is a help button for
further explanation of the displays (e.g. map legend).
There will be a simple action (button press) to perform to signal a request for
help from the staff; and another at the end of each consultation, to cancel
any help request you had made and give feedback on how helpful the consultation
was.
Web site logical path:
[www.psy.gla.ac.uk]
[~steve]
[grumps]
[this page]
[Top of this page]