21 Dec 1996 ............... Length about 2200 words (15000 bytes).
This is a WWW version of a document. You may copy it. How to refer to it.
To fetch a postscript version of it to print click this

New guises for recurring problems in documentation

(Commentary on Hallgren)


Contents (click to jump to a section)

by
Stephen W. Draper
GIST (Glasgow Interactive Systems cenTre)
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve

Preface

Commentary invited by Barbara Mirel. To appear in The journal of computer documentation vol.21 no.1 pp. 15-18 1997 Commentary on C.Hallgren "Using problem analysis to support decisions and planning in complex processes".

Introduction

Hallgren offers three vivid examples of problems with contemporary software and its documentation, together with the documentation solutions he designed. He also argues three analytic points: that these are complex problems, that improved documentation design should be based on "problem analysis" as well as task analysis, and that a key feature of his solution is providing some conceptual explanation, and not just the shallow procedural instructions characterising most documentation, and particularly online help. I will discuss all of these mainly from the point of view of the minimalist or action-centered approach to documentation: to what extent do Hallgren's cases and points constitute a challenge, and to what extent are they in fact consistent with that approach?

Complexity and contemporary problems in computer documentation

Hallgren describes three specific problems created by contemporary software taken with its normal documentation, and convincingly tells us that it baffles typical users. These are (a) a printout problem with QuarkXpress, (b) a quality problem with scanned-in color images in PhotoShop, and (c) a problem in setting up a network connection from new PCs to a server via another server. He also gives passages of his new documentation, which do seem likely to solve those problems.

Each of the cases is in effect presented in terms of the symptom or problem a user would hit (a printout with page layout problems, poor looking images, failure to connect to a server), and then the explanation for it appears within the text proposed as a solution in the form of new documentation designs. In each case, the symptoms amount to a problem because the existing system does not give usefully diagnostic messages. Each problem is in fact, it turns out, a setup problem i.e. caused (from the machine's point of view) by the failure of the user to perform some initialising actions correctly. Thus because they are setup problems, the missing user actions are not part of the main task procedures, and need to be done only once (a) per project setup, (b) per day, (c) per machine installation respectively.

Hallgren calls these problems "complex", but that is in my view misleading. Documentation is only necessary for things that users would otherwise find difficult or impossible; but conversely, if the documentation is effective then users will not find the task difficult. While reading his descriptions of the problems, I was convinced that as a user I would find them baffling, and thus "complex" in a subjective sense; but after reading his solutions, I felt they were easy to understand and that I could probably now solve them easily. This is, and always has been, the characteristic of problems posed by computer systems and solved by good documentation design. If you look at accounts of the problems posed by word processors in the past (e.g. Mack et al., 1983; Carroll, 1990, ch.2), you get essentially the same impression.

The only other aspect of "complexity" mentioned by Hallgren is that the user procedures might involve many steps, and might involve branching. These of course make writing effective documentation more difficult, but have always been part of typical tasks. The minimalist approach to documentation at least (e.g. Carroll, 1990) has always addressed this. It involves realising that documentation cannot limit itself to describing physical actions (e.g. key presses) but must also describe how users can recognize whether an action has succeeded, and so "branch" or otherwise react to the machine's state.

The complexity of contemporary problems seems to me, for the user, no different from those of 15 or 20 years ago. However because many aspects of the user interface have improved, the problems now typically appear in different places as Hallgren vividly portrays. Thus while old problems might have been to do with operating a command line interface for regular tasks, now the problems occur in the less heavily used and apparently less well designed parts of the user interface: the setup procedures. Similarly with respect to conceptual problems: whereas then a typical user was ignorant of the need to type in white space characters (spaces and linefeeds) to move the insertion point (why couldn't you just point to the bottom of the screen and type in there?), today the typical user is unaware that you need to load different fonts for screen and printer, to calibrate scanners frequently and to calibrate them differently for paper and for transparency originals, or to specify a net address in a different format just because it will pass through another net (my street address is not written differently on a letter posted in Italy than on a letter posted in Scotland, so why is a net address written differently depending on where it will be first used?). The question is, can old solutions be successfully applied with or without adaptation to solve the contemporary form of problem, and where do Hallgren's proposals for solutions fit?


Problem analysis and a user-centered approach

Hallgren's second analytic point is that documentation should be designed based on "problem analysis" rather than task analysis. Although he doesn't give explicit definitions, it seems that this means that instead of relying on an analysis of the procedure an error-free user would perform, the documentation designer should start by discovering what issues cause users the most trouble, for instance by consulting help desk statistics. The resulting documentation then addresses real problems that real users have. This is entirely consistent with the approach underlying minimalist documentation, but adds a useful suggestion about how to proceed. Although not highlighted by the name "minimalism", the fundamental approach behind minimalism is a user-centered one in the sense of basing the approach on usability testing of the documentation, and consequent iteration of the design (Draper, in press). However the literature to date on minimalism usually describes creating an initial design for documentation based on the designer's estimates of what might be needed, followed by observing users with this first version. Hallgren's suggestion of studying the problems with previously existing documentation is obviously sensible. It is furthermore consistent with an approach recommended in designing computer based training or indeed any pedagogical design, of studying common "misconceptions" that learners have with that topic, and designing new material in part specifically to address them. (Laurillard, 1993, argues for this, and recommends the phenomenographic method developed by Ference Marton for this purpose.) In fact following Hallgren's suggestion yields two kinds of information: the most frequently asked topics tell the documentation designer what topics to include in a new manual, while the advisers will be able to say what points and what ways of expressing them have turned out to be most effective in answering the enquiries, and these can be used to suggest the first draft for the new manual entry.

Another salient connection between Hallgren's approach and the minimalist approach is the importance of error recovery material. Although the term "minimalism" suggests stripping out material, in fact the deeper principles are user testing and organising documentation to support action (Draper & Oatley, 1992; Draper, in press). Thus, as Hallgren notes, minimalism always insists on including error recognition and recovery material, even though many manuals omit them. It is noteworthy that Hallgren's examples are presented in terms of a problem or "error" that a user would encounter, and then follow that with what a user would next have to understand and do to overcome the problem. At least as presented in the paper, his solutions all consist of additional error recovery material omitted from the original documentation: exactly in line with one of the main features of the minimalist approach.

Including explanations

Hallgren's third argument is for the need for conceptual explanations, and this is definitely not part of the main minimalist prescriptions. On the contrary, minimalism emphasizes how users tend to press on with the task in hand, and are liable to skip descriptive material. It does not rule out explanatory material, but its approach certainly tends against its use. Hallgren thus raises an issue that is important, but not resolved in the minimalist literature to date.

It is interesting however to notice the details of Hallgren's proposals, and what he does not himself resolve. As noted above, in all three cases described, the problem in fact is that users failed to carry out necessary setup procedures. It might seem therefore that the manual should present these setup actions as part of standard task procedures. Hallgren implies however that his explanations, and the setup procedures associated with them, are only to be presented to the user as error recovery procedures consulted after problems have occurred and the user is in trouble. This is probably not an accident. The minimalist work tells us what many have observed for themselves, that typical users may not consult a manual until they are in trouble, and will certainly not read descriptive or explanatory passages without having a special reason to do so. Running into a serious problem may be the necessary reason. Note too that each of these three problems is assoicated, not with a short time scale problem (e.g. which menu item to try next), but a relatively long time scale: getting a printout, scanning in a picture, setting up a new PC to connect to a network. Users may perhaps ration their patience for reading explanations in proportion to the time it takes to go round action cycles of this kind (the time it will cost them to try and fail again). It would require considerable empirical work to explore this point properly, but a minimalist can take from Hallgren's paper the suggestion that the place for explanations may be mainly inside error recovery sections, where readers have been given the motivation to read and try to understand explanations by having run into a serious practical problem, and that the length of acceptable explanations may be proportional to the duration of the associated action cycle.

There remains the issue, again needing considerable investigative work, about the optimum form for Hallgren's additional material. The three samples he gives mix explanatory conceptual explanation with procedural information. This works in the context of the paper, where readers are bound to read all the material in the sequence he gives it. While a user is still seeking a general explanation for a major problem, that will probably also work in a manual. But as the minimalist work has explored at length, a discursive format does not work so well as users try to carry out the instructions, switching frequently between text and the machine. Thus there is more work to do to determine how best to structure and format explanation plus recovery material of the kind he proposes.

Finally, however, there is an implicit aspect of this that is even more challenging to the minimalist approach as so far developed. Hallgren successfully evokes in the readers of his paper, in me certainly, first a puzzlement and desire to understand, and then a satisfied feeling of having understood what is behind each problem. His documentation is thus addressed to satisfying a powerful user desire. Minimalism is focussed on supporting successful user action (and the powerful feelings of satisfaction that brings to users), not to satisfying a desire to understand. Although the overall aim of being user-centered could regard both as desirable, much of the experience of minimalist practice is of cases where the route to successful action is not through telling users the answers to conceptual questions. Furthermore it has been demonstrated even outside the field of minimalism that a feeling of conceptual understanding can be no guide at all to whether someone can perform the task successfully: Kieras & Bovair (1984) showed that teaching subjects the theory behind a procedure might give us the feeling of comprehension, but left subjects no better able to perform the procedure. Conversely some of the papers Hallgren cites on display based competence show that users may perform procedures effectively with little memory or undestanding of what they do. So the lasting challenge I find in Hallgren's paper is to consider further the relationship between these two user desires, that are sometimes in conflict: to understand and to succeed at the concrete task.

References

Carroll J.M. (1990) The Nurnberg funnel: designing minimalist instruction for practical computer skill MIT Press: Cambridge, Mass.

Carroll J.M. (in press) (ed.) Reconstructing Minimalism MIT Press: Cambridge, Mass.

Draper, S.W. (in press) "Practical problems and proposed solutions in designing action-centered documentation" in Reconstructing Minimalism (ed.) J.M.Carroll MIT Press: Cambridge, Mass.

Draper S.W. & Oatley K. (1992) "Action Centered Manuals or Minimalist Instruction? Alternative theories for Carroll's minimal manuals" in P.O.Holt & N.Williams (eds.) Computers and Writing: state of the art ch.15 pp.222-243 Oxford: Intellect Books; and Norwell, MA: Kluwer Academic Publishers

Kieras,D.E. & Bovair,S. (1984) "The role of a mental model in learning to operate a device" Cognitive Science vol.8 pp.255-273

Laurillard, D. (1993) Rethinking university teaching: A framework for the effective use of educational technology London: Routledge

Mack R.L., Lewis C.H., & Carroll J.M. (1983) Learning to use word processors: problems and prospects ACM trans. office information systems vol.1 254-271.