Last changed
12 Nov 1997 ............... Length about 2200 words (14000 bytes).
This is a WWW document by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/dom.html.
You may copy it. How to refer to it.
Researching Domains
by
Stephen W. Draper
GIST (Glasgow Interactive Systems cenTre)
Department of Psychology
University of Glasgow
Glasgow G12 8QQ U.K.
email: steve@psy.gla.ac.uk
WWW URL: http://www.psy.gla.ac.uk/~steve
Requested by Nigel Birch, an attempt to write out an opinion of mine in
favour of grants to study and identify problems, rather than solve them.
This document is a personal view. It proposes putting a significant
emphasis in HCI research taken as a whole on studies of "domains": of work
domains, and other ways of defining problem areas.
Increasingly, the success of computer applications is being determined,
not by mastering the technology by ever better programming techniques and
structures, but by improving the way in and the degree to which the computer is
integrated with the work domain it is being applied to, and succeeds in serving
that domain as a whole. This is affecting computer science as a whole, but is
of central importance to HCI.
There are numerous symptoms. Bank cash machines are not referred to by the
public as computers at all, but viewed as a specialist device: already implying
an integration. They are in direct competition for users with the human
tellers, often just inside the building. On the one hand this demands higher
design quality because the users are not forced to use them, but on the other
hand it also means that not every function need be covered by the machines
because the humans offer an alternative mechanism. Both what counts as success
and its achievement depend on the ensemble — the combination of machines
and human tellers. In a study some years ago (Baxter & Oatley, 1991)
comparing the interfaces of two rival commercial spreadsheets, it turned out
that there was no difference whatsoever between the spreadsheet designs in how
long it took to learn them, but a significant difference between users who had
never used a spreadsheet and those who had used some other one. Clearly a lot
of the learning burden was not in learning to operate the user interface, but
in learning the domain of spreadsheets in general. In studying current
problems with user manuals (documentation), more and more the things users
really need help with are not the operation of the interface, but the
presentation of the work domain and how the software relates to it. For
instance, to support people through an improved computer tool in keeping the
accounts for a simple personal bank account, you probably need not only to
support both cleared and commitment accounting, but to explain to them this
concept, and the importance of tracking both views of the "same" account.
(This refers to the distinction between the bank's view of the net amount in
the account given transactions that have gone through up to this moment,
versus the practical view of what you have left to spend given all the
commitments such as standing orders and cheques sent out in the mail you have
already made.)
To some extent at least, taking work domain studies seriously requires
different techniques than those most often employed in HCI (at least until
recently), and may involve observational and interpretative practices from
sociology rather than psychology.
There is a tradition for such an emphasis (e.g. in ergonomics, in John Long's
group at UCL, and in Rasmussen's (1994) work), but there it has been usually
associated with big workplace projects such as the control rooms of big
electricity generating stations. It should now be given wider attention within
HCI, as in fact such domain issues are pervasive. Currently (1997) the
sociologists working in HCI are doing so almost exclusively either in CSCW,
where the name of the field puts the central focus on social interaction and
hence legitimates social observations, or in air traffic control, which
continues the longer tradition of "big" applications and in any case is itself
a kind of CSCW. The suggestion here is that we should take seriously extending
this perspective to all domains.
In fact there is another aspect I wish to bring out. Existing studies
emphasising workplace studies are usually justified because it is clear in
advance that social interactions are central to the application (as in ATC and
CSCW). Thus including workplace studies in such cases in fact extends the
usual simple story taught to software engineering students in which the client
knows they have a problem and hires a team to solve it; requirements gathering
is essentially elaborating the problem definition of the client; specification
and implementation are further elaborations. In reality however a proper
requirements study may discover that the problem is quite different from what
the client thought, and one implication is that a very different set of
resources and a different team may be needed to solve it. One example I
recently encountered was the design of web pages for departments (Draper;
1997). These had been seen as essentially a low grade design job (HTML is a
simple language, so hire a low skill programmer). But a systematic study
brought out the fact that key requirements include keeping any page up to date
(so storing the information in a data base which generates pages automatically
is cruical, not hand-maintained pages), and building on existing human work
practices for gathering and publishing the information is equally important as
otherwise whole new (and unaffordable) jobs will be created. This vision of
requirements gathering, not as a first step in a programmers' job but as a
separate job needed in order to define (or rather to discover) the problem, is
in fact less unfamiliar in commercial practice, where it is quite common to
have one contract to do the requirements gathering and another to do the other
stages. But it also has research implications. It could be appropriate to
fund research to study and publish problems or domains: this is not only
worthwhile, but it can lead to surprises, thus rendering it appropriate for
publicly funded research as the benefits cannot be clearly predicted in advance
(if they were, then the interested parties would pay for it). The big benefits
as seen afterwards will be cases where clients in the domain did not identify
their problem, or not correctly: doing such a study could benefit users and
their suppliers alike in the domain studied.
One example would be case of web pages for a university department or
medium sized enterprise (Draper; 1997). HCI work has usually, so far, focussed
on page design, navigation problems, or technical correctness (e.g. broken
links). But studying users of such pages in fact suggests that more important
bottlenecks are firstly ensuring that end users' information needs are
satisfied (given that such users seldom meet the page's authors who thus get
very little feedback), and secondly setting up human procedures that will
supply the information, check its accuracy, and keep it up to date. Page
design may be unimportant compared to the issue of whether the information the
user wants is there at all, and whether it is up to date. Many clients hire
specialist designers once to "do their web pages", mis-identifying what their
problem is, and getting web sites that go rapidly out of date. A study
establishing whether this analysis is correct, and grounding it solidly on
data, would serve everyone.
Another example might be the use of relational databases. This technology is
now extremely mature, and available very cheaply. Yet it still seems to lack
the penetration into practice I would expect it to have. Why do not we all
hold our address books, lists of references, CVs, lists of all kinds in
databases? My suspicion is that there is still not a good bridge for ordinary
users to analysing their application into data entities i.e. it is the data
model design that is the bottleneck. But this is only a guess: evidence would
be much better, and could then focus future development in the database
field.
Too much HCI research currently is technique-based: developing new
interaction techniques for their own sake (hoping for plausible applications
later), and too little studying problems without having a solution technology
in advance.
The proposal, then, is to encourage some research projects that study work
domains and problems from a user perspective as a whole project in themselves.
They would aim to analyse and publish this analysis, which might then
stimulate a different kind of research from those who conceive of an
interesting solution to the problem so defined. Although the ideal research
team for such domain research might be inter-disciplinary, as that would best
guarantee effective communication of the results, it should still be fruitful
for some studies to be done almost entirely by specialists such as
ethnographers who would not normally expect to perform HCI projects by
themselves. This might be desirable, as not only is it hard to coordinate
inter-disciplinary teams, but bigger teams need bigger problems to be
worthwhile: so small studies would not be done, and the focus would remain on
big applications like ATC. Promoting a category of "problem domain studies"
would emphasise that small focussed studies could be useful, and would support
the idea that analysing a problem (i.e. doing requirements gathering for
little understood domains) is a job in its own right, requires its own kind of
expertise, and should not continue to be seen as part of "design" where that
is understood only as the production of artifacts. Just as for centuries
medical research was essentially a naturalist's domain — describing
diseases not attributing causes much less cures — so there may well be
value in encouraging in the HCI field the study of problems and their domains,
and regarding the publication of their description and analysis as not only
prior to but independent of the need to propose a fix.
From the viewpoint of the organisation of EPSRC, this proposal has
several aspects.
First of all it is advocating the active study of places that research and
development might later be socially or commercially valuable, instead of
waiting and hoping that spinoffs will come. HCI is one of the few places where
this is justified, because understanding the interaction of computer technology
with work practices is ever more important in this field.
Secondly, while workplace studies could be done by a wide variety of people,
some of these might come from social sciences more often funded by ESRC.
The "effectiveness" theme might be reworded towards less emphasis on
benchmarks and more on studies of how work is or is not supported in reality
by software: both the domain studies argued for above, and case studies of
effectiveness.
Domain studies in one sense are very applied, rather than basic research.
But on another sense, they are basic: in that they are not directly related to
developing an application but at their best might lead to new kinds of
application in the future (by identifying new needs).
- Workplace studies of how the design and use of computers interacts
with their human work context and its social interactions is increasingly
important.
- It is time to make more use of sociology in the multidisciplinary approach
to HCI.
- We should encourage, rather than discourage, separate studies involving this
aspect. This is against older views in software engineering that this is just
part of requirements gathering which is the first stage of design; and against
the view in HCI that "design" is the whole business, and production of software
the only important activity or product. If companies must produce code to make
money, that simply strengthens the argument that publicly funded research
should concentrate on aspects that will necessarily be under-emphasised
commercially. But the fundamental reason why such studies should often be done
without immediate design results, is that they frequently change the whole
nature of what the design project should be. They are about identifying and
analysing the problem, without which designers are acting blindly.
- Such research projects would be about a domain: perhaps defined by a kind of
work, or perhaps by a kind of technology that seems to be underperforming.
They might produce a theory of it i.e. an explicit description of the work and
work practices in it; this would inform design aimed at supporting it, and
also the design of how such understandings could be conveyed through the
technology to users new to the domain as well as the software. They might also
produce a description of the key issues, of constraints and problems likely to
affect any design in that domain. Often they will identify problems not
correctly identified by users and designers. Publishing these analyses could
benefit users in the domain directly; and could lead to followup research
(probably by others with different expertise) in which solutions were
pioneered.
Baxter, I. & Oatley, K. (1991) "Measuring the learnability of
spreadsheets in inexperienced users and those with previous spreadsheet
experience" Behaviour and Information Technology vol.10 pp.475-490.
Draper (1997, Sept, 3) The problem of departmental web pages [WWW
document] URL
http://www.psy.gla.ac.uk/~steve/webdesign/web.html
Rasmussen,J., Pejtersen,A.M. & Goodstein,L.P. (1994) Cognitive
systems engineering (Wiley: New York)