Last changed 27 May 2003 ............... Length about 900 words (6000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/ilig/feedback.html.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [EVSmain] [this page]
Question design: [one question] [purposes] [contingent sessions] [feedback] [whole sessions]

Feedback to students

(written by Steve Draper,   as part of the Interactive Lectures website)

While the presenter may be focussing on finding the most important topics for discussion and on whether the audience seems "engaged", part of what each learner is doing is seeking feedback. Feedback not only in the sense of "how am I doing?", though that is vital for regulating the direction and amount of effort any rational learner puts in, but also in the sense of diagnosing and fixing errors in their performance and understanding. So "feedback" includes, in general, information about the subject matter, not just about indicators of the learner's performance.

This can be thought about as levels of detail, discussed at length in another paper, but summarised here. A key point is that, while our image of ideal feedback may be individually judged and personalised information, in fact it can be mass produced for a large class to a surprising extent, so handset sessions may be able to deliver more in this way than expected.

Levels of feedback (in order of increasing informativeness)

  1. A mark or grade. Handsets do (only) this if, with advanced software, they deliver only an overall mark for a set of questions.
  2. The right answer: a description or specification of the desired outcome. Handset questions do this if the presenter indicates which option was the right answer.
  3. Diagnosis of which part of the learner action (input) was wrong. When a question really involves several issues, or combinations of options, the learner may be able to see that they got one issue right but another wrong.
  4. Explanation of what makes the right answer correct: of why it is the right answer. I.e. the principles and relationships that matter. The presenter can routinely give an explanation (to the whole audience) of the right answer, particularly if enough got it wrong to make that seem worthwhile.
  5. Explanation of what's wrong about the learner's answer. Since handset questions have fixed alternatives, and furthermore may have been designed to "trap" anyone with less than solid knowledge, in fact this otherwise most personal of types of feedback can be given by a presenter to a large set of students at once, since at most one explanation for each wrong option would need to be offered.

The last (5) is a separate item because the previous one (4) concerned only correct principles, but this one (5) concerns misconceptions, and in general negative reasons why apparent connections of this activity with other principles are mistaken. Thus (4) is self-contained, and context-free; while (5) is open-ended and depends on the learner's prior knowledge. This is only needed when the learner has not just made a slip or mistake but is in the grip of a rooted misconception -- but is crucial when that is the case. Well designed "brain teasers" are of this kind: eliciting wrong answers that may be held with conviction. Thus with mass questions that are forced choice, i.e. MCQ, one can identify in advance what the wrong answers are going to be and have canned explanations ready.

Here are two rough tries, applying to actual handset questions posed to an introductory statistics class, at describing the kind of extra explanation that might be desirable here. Their feature is explaining why the wrong options are attractive, but also why they are wrong despite that.

Example1. A question on sample vs. population medians.

The null-hypothesis for a Wilcoxon test could be:
  1. The population mean is 35
  2. The sample mean is 35
  3. The sample median is 35
  4. The population median is 35
  5. I don't know
Why is it that this vocabulary difference is seductively misleading to half the class? Perhaps because both are artificial views of the same real people: the technical terms don't refer to any real property (like age, sex, or height), just a stance taken by the analyst. And everyone who is in the sample is in the population. It's like arguing about whether to call someone a woman or a female, where the measure is the average blood type of a woman or of a female. And furthermore because of this, most investigators don't have a fixed idea about either sample or population. They would like their ideas to apply the population of all possible people alive and unborn; but know it is likely that it only applies to a limited population; but that they will only discuss this in the last paragraph of their report, long after getting the data and doing the stats. Similarly, they are continually reviewing whom to use as a sample. So not only are these unreal properties that exist only in the mind of the analyst, but they are continually shifting there in most cases. (None of this is about casting doubt on the utility of the concepts, just about why they may stay fuzzy in learners' minds for longer than you might expect.)

Example2. Regression Analysis: Reading versus Motivation

PredictorCoefSE CoefTP
Constant2.0741.9801.050.309
Motivati0.65880.36161.820.085
The regression equation is Reading = 2.07 + 0.659 Motivation
S = 2.782     R-Sq = 15.6%     R-Sq(adj) = 10.9%

Which of the following statements are correct?
a. There seems to be a negative relationship between Motivation and Reading ability.
b. Motivation is a significant predictor of reading ability.
c. About 11% of the variability in the Reading score is explained by the Motivation score.

  1. a
  2. ab
  3. c
  4. bc
  5. I don't know
There was something cunning in the question on whether a correlation was significant or not, with a p value of 0.085. Firstly because it isn't instantly easy to convert 0.085 to 8.5% to 1 in 12. 0.085 looks like a negligible number to me at first glance. And secondly, the explanation didn't mention the wholly arbitrary and conventional nature of picking 0.05 as the threshold of "significance".

For more examples, see some of the examples of brain teasers, which in essence are questions especially designed to need this extra explanation.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [EVSmain] [this page]
[Top of this page]