Web site logical path: [www.psy.gla.ac.uk] [~steve] [this page]

Supporting use, learning, and education

Commentary on Guzdial's "Supporting learners as users"

Stephen W. Draper
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve/

Contents (click to jump to a section)

• Introduction
• Learning, doing, and software
• Guzdial's work considered as software documentation
• Guzdial's work considered as educational software
• Conclusion
• References

Introduction

The fundamental starting point for Guzdial is that users and learners typically have a big goal stack: they have high level, long range aims to do with their work domain, each subdivided many times and ultimately resting on a large number of small actions (such as moving the mouse, picking a menu command, etc.). If the small actions are automatic, fully learned ones, then the theoretical existence of the goal stack is of little consequence. However if those small actions are themselves problems requiring conscious attention and a search for information, as they may well be for users new to the software, then the whole activity becomes liable to breakdown and failure. Guzdial points out that learners, particularly learners in the education system, will have particularly large goal hierarchies because their top level goals are to do with passing a course and learning concepts, and even the highest goals to do with the software are already a significant distance down the stack; and furthermore in many cases the software as well as the educational goals will be new to the learner, adding to the simultaneous learning burden.

Learning, doing, and software

1. They are only attempting to learn the software in response to an immediate larger goal (e.g. a demand at work) for which the software is a means to that end. In this respect, Guzdial's students are typical of most users that documentation writers must try to support: both intrinsically impatient and under pressure from extrinsic goals. The only common exception is users on training courses, whose main support for learning is exactly that the immediate pressure of extrinsic goals has been lifted by secluding the user away from their work on the course for a few hours or days. Interestingly, such courses usually employ special documentation to suit these circumstances: normal documentation must support learning with a big goal stack.
2. Using the software for a domain project is the best and easiest way to understand software anyway (i.e. in relation to the tasks it supports). This simply illustrates the profound connection between learning and doing (an argument developed in Draper, 1998). In fact it is likely that it is not possible for humans to do one without the other: we learn by doing, and whenever we do something we learn a bit more about how to do it. Guzdial clearly appreciates this constructionist argument for large educational goals, but it applies equally at the level of user interface actions and documentation. There is of course a significant difference between users who are trying to accomplish work goals and learn only as a side-effect (although actually this gain in experience will make them more valuable workers), and users who are trying to learn and acting only as a means to learning by doing. Nevertheless in both cases users are both doing and learning. Furthermore Guzdial's students are exactly like most other users with respect to the software, in that their main goal is not to learn how to use it but to do (to use the software for some other end).

The challenge for supporting them is also the same, even though this is not very well recognised by documentation writers: good support should not only support the low level interface-use goals but also allow the users to learn about the domain the software supposedly supports. In a word processor this would mean supporting users in learning what a well formatted letter is like, how to write sentences in a good style, how to structure a book. Software, although less often the documentation, now increasingly attempts this, although most users have learned this domain knowledge elsewhere. With spreadsheets, a user must learn not just the interface but what a spreadsheet could be used for. Experimental evidence (Baxter & Oatley, 1991) suggests that a substantial part of the learning that a new spreadsheet user does is not of the interface but of doman knowledge common to all spreadsheets. With a simple accounting package (e.g. for managing one's personal finances) this is even more obvious: the software will be of no benefit unless the user learns practices and concepts they may not have learned anywhere else (e.g. the distinction between transaction and commitment accounting). All software and its documentation must increasingly accept the responsibility for task domain instruction and support, as argued in Draper (1998). This is essentially the problem Guzdial identifies for educational software, and argued for in Soloway et al. (1994).

Guzdial contrasts three kinds of users: expert-as-user, learner-as-learner (e.g. using Computer-Assisted Instruction), and learner-as-user. In fact the first two are really the same: CAI only works smoothly when it is careful to employ only interaction techniques that those users (students) are already familiar with. When that is the case, both the first two categories refer to users who are expert in the software and using it to pursue their extrinsic work goals, which for students are educational learning goals. After all, experts do not typically use software for its own sake but to achieve work beyond the software. His use of the term "learners" in these category labels is infelicitous as it is ambiguous between educational learning (his meaning) and learning to use the software i.e. learners as contrasted to experts in the software (which is the main subject in his paper). All support, both conventional documentation and the support methods he reports on, aims to support users in learning the software better.

This terminological problem is related to the claim that Guzdial, referring to Soloway et al. (1994), makes that "traditional user-centered design ... tends to focus on expert users". That paper mainly talked about learning or expertise in some work domain, and whether software could be designed to support learning about the domain. However taking the statement at face value as about about computer users, and learning and expertise in the software tools themselves, it is certainly wrong: true user-centered design (Norman & Draper, 1986), which is a reaction against the traditional design focus on the expert users that designers most easily identify with, begins by identifying the target user group and designing for them (see introductory HCI textbooks e.g. Newman & Lamming, 1995), and the single most obvious user classification is into novices and experts. If you are designing for aircraft pilots or travel agents then you know that all your users will be working only after special training i.e. will be experts, whereas if you are designing public information kiosks or emergency exits then you know that almost all your users will be first time users. A slightly more sophisticated approach might distinguish between usability costs for experienced users, the "guessability" costs for first time users, and the learnability cost for a user moving between the two (Jordan et al. 1991). Since experts have to start by learning, most applications involve a mixture of these costs, but it remains true that different applications put very different relative weights on these contributions.

Furthermore educational users are again not any different in this respect. Most CAI software will only be used once or twice by each student, so all are first time users, while some of the most important software in education today is basic word processing, spreadsheet, email, and web browsing which students may be expected to learn early on and use throughout several years of study. In computer science courses this could also apply to a programming language if one is selected as standard throughout a degree programme. In these cases, the student users are "expert" not first time users of the software. All of this also applies to documentation. If the target users of the documentation are experts, then a reference manual may be satisfactory, while if the target users are first time users of the software, then a Minimal Manual approach is much more likely to be appropriate.

Guzdial's approach for his student users can be discussed from two perspectives: as documentation techniques to support any software user, or as educational techniques. His main focus is on educational ideas and these are interesting in their own right, but it is also interesting to apply them to documentation, both because (as he shows) education is an interesting software application domain and to judge whether they suggest interesting approaches to learning software i.e. the documentation domain itself.

Guzdial's work considered as software documentation

• Consistency in interface design, exploiting methods the users have already learned elsewhere. Such interfaces avoid introducing new learning problems by re-using existing user expertise.
• Guessable user interfaces: where users can pick the correct action even the first time.
• LBE: supporting learning by exploration.
• Scaffolding: tutors take over or otherwise guarantee some or all functions
• Training wheels: modify the software to disallow tricky or complex tasks and subtasks that tend to derail novice users. (A form of scaffolding by the software.)
• Setting simple exercises for training purposes. (A form of scaffolding by syllabus.) In a simple form, this might be suggesting a realistic task but one selected to avoid problems for novices; while in the limit it becomes separate artificial training on each component subskill.

Observations of users show that LBE is probably the leading source of information for new users today (how often do YOU read the manual or online help instead of just trying something out?). This is no accident: the direct manipulation properties of today's mouse, menu, and window interfaces such as immediate visible feedback for all user actions, reversible/undoable actions and so on (Hutchins et al. 1986) are in part deliberately designed to support this mode of user learning (in contrast, say, to the original Unix interface with no feedback, no menu prompts to show the available actions, and many dangerously irreversible actions). Smalltalk, too, arose in part from a desire to bring this property of visibility and hence LBE to programming (Kay, 1993), thus making programming accessible to children and indeed all types of user, although it is debatable how successful this part of the vision has been (so far).

Most CAI software uses the first two techniques of consistency and guessability (e.g. by restricting itself to simple point and click interaction or a few menu choices) so that users can focus on the educational goals with little or no additional learning of the software itself. Syntax-directed editors in programming languages embody a form of training wheels approach, and Emile embodied another. Minimalist documentation in effect tries to fit in closely with user attempts at LBE, supplementing the user interface with additional printed information. In contrast Guzdial's approach is to provide web resources that fulfill some of the function of scaffolding without using human (or artificial intelligence) tutors. From a learning perspective, LBE and scaffolding could be seen as closely related: after all, the point of parents providing scaffolding for their child or a master for an apprentice is in both cases so that the learner can (with help) learn by doing, which is equally true in LBE. For the provider, however, they are quite different kinds of support.

Because what Guzdial's approach supplies is written resources, we could view them as a kind of documentation; but they are designed and created in a different way: largely by the (student) user community itself. They are a form of automatic or self-creating documentation. In a naive form of this idea, anyone needing to know something would go to the trouble of discovering the answer and recording it in a place where it could in future be re-used with less effort by themselves and other users. The documentation would thus consist only of information that had been needed by a real user at least once (rather than guessed by someone else to be required), and the effort of creating it would be spread across the community that needed it. A primitive form that is commonly seen is the FAQ (frequently asked questions) document, created by a designers on the basis of questions asked of them by other users. Like documentation created on the basis of help-desk data, this also has the feature that all the content has been needed by real users, although the work is done by the designers not users. The best developed version of the self-creating documentation idea may be Ackerman's Answer Garden (Ackerman & Malone, 1990; Ackerman & McDonald, 1996; Ackerman & Palen, 1996). In this software, information items are organised in a hierarchical (tree) index. Users with a query search the tree and either find what they need, or else issue a query from the nearest point of the tree. This is automatically routed to an expert responsible for that part of the tree, and the reply both goes to the user and is stored in the tree for future re-use. Guzdial's approach has similarities to this, and it is an approach that could be applied to all documentation creation.

The other important feature of Guzdial's approach, however, is the use and re-use of substantial examples of past work. In that way, his approach is related to the literature on software re-use, and the issues involved in making that effective. In learning a programming language, as opposed to an application package, such examples are very powerful indeed for new users. In fact a good set of examples but no other (explanatory or procedural) help may well be more useful than any other kind of documentation without examples. Again, this is a general approach to documentation that could be applied widely.

It is interesting and significant that Guzdial identifies indexing as a central problem in his approach. It is also the central problem in software re-use, in Answer Garden (where the index tree tends to have to be manually rebuilt periodically as the collection as a whole changes), and indeed in minimal manuals (see Draper, 1998). One fundamental issue is that although self-creating documentation methods may generate items for free, organizing them for future access turns out to be an independent task that is equally important, but must be done by the provider after all. (Unless, that is, some new idea is developed for a self-indexing method, although in application contexts like the WWW perhaps generic information retrieval software may be a significant help.) The second fundamental issue is that of description: new users need old material to appear somehow under their personal descriptions of new needs. In software re-use, old units need somehow to be recognisable as related to new applications. In minimal manuals, a command (say "cut and paste") has somehow to be recognisable to users in their own task terms (say "moving text") before they have learned that interface's command terminology. Nevertheless, although maintaining the index has emerged as a significant new problem associated with these approaches, without any good solution so far emerging, it does not seem to destroy their value but simply to mark a need for further work. Guzdial's experience in this respect seems consistent with others'. Indeed, perhaps he has identified an important new aspect. He reports that his student users of STABLE wanted, not to access one or two best cases in detail, but to compare them. So just as the way forward for documentation may not be for a designer to produce just the right items but to collect user items, so the way forward for the index may not be for the provider to produce just the right index pointing to a single best item, but rather for it to produce a good subset and support user comparisons within that subset.

From a documentation viewpoint, then, Guzdial's approach concerns two techniques worth general consideration: self-documentation (constructing documentation by collecting questions and answers only from users engaged in applying the software), and example-based documentation (essentially libraries of past uses of the software). Whether these techniques are adopted will depend not only on their effectiveness in supporting users, but also on perceived obligations i.e. whose job it is thought to be to create the documentation.

Guzdial's work considered as educational software

Although Guzdial does not offer a justification for this fundamental feature of his approach, there is indeed a standard counter-argument supporting it (and backed by experience) in the educational literature. Learning through "real-world" problems, i.e. tasks with a close correspondence to the work domain the learner is being trained for, as opposed to training on component subskills that the learner is assured will be relevant in a few years time, is both much more motivating and also is direct training on the application tasks that the student must eventually master. (In other words, it trains them on the goal stacks they will ultimately have. When this is not done, you see graduates with many of the necessary skills yet unable to connect them to and mobilise them for practical applications.) On this analysis the analogy with introductory and advanced calculus is a false one. There is a pre-requisite dependency, but it is that between a tool and the reason for its existence. The educational arguments are firstly that both what it is used for as well as how it is operated must be taught (and that to teach only one is to omit essential learning content), and secondly that this is as important to student motivation during learning as it is to the general motivation or rationale for providing such courses at all.

This approach is now being adopted by some leading medical schools in both the USA and the UK in the form of "problem based learning" (PBL) (Boud & Feletti, 1991). Instead of training medical students by lectures for some years, and later moving them on to some "applied" experience in the wards, under PBL teaching from the start consists of splitting the students into groups, and setting each group a sequence of problems (typically on a 2 week cycle) based on a real life application (a description of a patient presented for treatment). The group first decides what they need to find out, and then accesses resources (e.g. in the library) to satisfy their self-determined learning subgoals. Guzdial's approach, then, is related to important strands of contemporary educational development.

Indeed the issue of whether to learn through subskill training or through whole application tasks is a very general one. For instance in teaching children to read it corresponds to whether to teach them "phonics" i.e. how to use letters to decode novel words, or only through reading interesting material where context, storyline, and pictures all help to contribute to understanding. Any course with significant project work is adopting the whole task approach to some extent: PBL is unusual mainly in that it attempts to deliver a whole curriculum this way and by contrast with the methods it replaces. In documentation, as noted, it corresponds to the minimalist approach of "getting the user started fast" and of constructing documentation that helps users with work tasks rather than only with explaining isolated commands. From this perspective, Guzdial is supporting his educational users in the spirit of minimalist documentation (no prior training on the software but starting immediately with, and wholly organized around, trying to get domain work accomplished), but using distinctly different techniques.

Given this educational decision to go for whole task teaching, Guzdial attempts to support it by a form of self-creating and example-based documentation. These too have educationally positive features (that may make them more appropriate in an educational application than in other cases where documentation is seen as the obligation of the manufacturer). At least in the CoWeb case, these include peer collaboration, students identifying their own needs and organising their own solutions, and teachback. These are thought to be important not only for developing a learning community, and enabling students to judge their own progress by being in contact with others' problems and successes, but also because a good way to learn something is to teach it to someone else. (Although there is much less benefit from having to teach the same item again and again, the first time at least it requires processing the material in a new way which is associated with deeper understanding and longer retention.)

Guzdial in fact probably understates the value of his work. He stresses that it is significantly cheaper than the approaches of Harel and Kafai because it replaces labour-intensive human tutor support by student-generated and computer-delivered material. However that process probably has additional inherent benefits for the students who generate it. Furthermore it adds to our research knowledge by reducing the theoretical concern that those human tutors might have been performing some other function (such as increasing learner motivation by all the personal attention) than the one ascribed to them.

Conclusion

References

Baxter, I. & Oatley, K. (1991) "Measuring the learnability of spreadsheets in inexperienced users and those with previous spreadsheet experience" Behaviour and Information Technology vol.10 pp.475-490.

Boud, D., & Feletti, G. (1991) The Challenge of Problem Based Learning (London: Kogan Page)

Carroll J.M. (1990) The Nurnberg funnel: designing minimalist instruction for practical computer skill (Cambridge, Mass.: MIT press).

Draper, S.W. (1998) "Practical problems and proposed solutions in designing action-centered documentation" Minimalism beyond the Nurnberg funnel (ed.) J.M.Carroll ch.13 pp.349-374 (MIT Press: Cambridge, Mass.)

Hutchins, E.L., Hollan, J.D., & Norman D.A. (1986) "Direct manipulation interfaces" in D.A.Norman & S.W.Draper (eds.) ch.5 pp.87-124 User Centered System Design (Erlbaum: London).

Jordan, P.W., Draper, S.W., MacFarlane, K.K., & McNulty, S.A. (1991) "Guessability, learnability, and experienced user performance" in HCI'91 People and Computers VI: Usability Now! (eds.) D.Diaper & N.Hammond pp.237-245 (Cambridge University Press: Cambridge).

Kay, A.C. (1993) "The early history of Smalltalk" in ACM SIGPLAN notices vol.28 no.3 pp.69-95.

Newman,W.M. & Lamming,M.G. (1995) Interactive system design (Addison-Wesley: Reading Ma.)

Norman D.A. & Draper S.W. (1986) User Centered System Design (Erlbaum: London).

Soloway,E., Guzdial,M. & Hay,K.E. (1994) "Learner-centered design: the challenge for HCI in the 21st century" Interactions vol.1 no.2 pp.36-48

Web site logical path: [www.psy.gla.ac.uk] [~steve] [this page]
[Top of this page]