Last changed 6 June 2003 ............... Length about 900 words (6,000 bytes).
This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/localed/rblack.html.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [localed] [this page]

Project on predictors of student performance in first year programming

This page is a few notes about a followup to Matt Roddan's project, being done by Rebecca Black.

Below are provisional plans, and not all the aspects described here will necessarily be acted on this year. The completed project report is: Black,R. (2003) Predictors of first year computing science student failure (Department of Psychology, University of Glasgow).

Main idea

The overall approach of both Matt Roddan's project and this one is to look for things that might give an early indication of a student being likely to do badly in the course overall. If we could find any such early indicators, then it might subsequently be possible to help prevent this and improve the performance of such students. We are in principle interested in a very wide range of possible indicators. We may use data gathered in different ways: some paper questionnaires, some questions asked in lectures using the handsets, and so on. Students will be informed about each data collection method separately: this page offers some overall information about the project.

The first idea in this particular project is to test a tantalising, but fragile, finding of Matt Roddan's: that the only factor with good predictive power (correlation 0.7) for whether a student would do well in the class exam on level 1, semester 1 computer programming was their own self-estimate of how well they understood the material.

While we are at it, we will also explore more potential predictive factors, taking Matt Roddan's explorations of these still further.

All such potentially predictive factors will be tested for correlations with "outcome measures" such as exam performance, course performance; and perhaps some extra measures of our own of how well students get on e.g. motivation, whether they feel interested, ... If we could discover early predictors of which students were most likely to do badly, we could alert them and their tutors in time to make a difference.

Aspects

Measure / investigate all of these:

Main issue

Weekly self-estimates by students of their understanding (see how soon they become predictive). Can students in fact privately tell how well they are going to do? Correlate these with marks in class exam (and lab exam); in January and in June.
Get tutors to estimate how well their students will do. Can tutors in fact privately tell how well their students are going to do? If not every week, then say at week 6 and 12.
Other things that might predict performance
Measure prior training and attainment in maths, general computing, programming. Measure this in some detail as the standard records are not necessarily fine-grained enough. In particular, learning about computers is NOT the same as learning to program; and learning programming may either be about learning the same language (and so the same vocabulary) OR might be analysed as having learned specific semantic (not notational) constructs: if-then, for, while, recursion, ....
Measure aptitude. If we can find an aptitude test. One point of this is that it may be a tougher contrast by which to judge the (so far small) predictive power of other variables. But it is also of interest in its own right.
Measure "Tinto" variables of "integration". Tinto's is the single most referenced theory in the research literature on student dropout and retention. Its central idea is that of "integration": it claims that whether a student persists or drops out is quite strongly predicted by their degree of academic integration, and social integration. These evolve over time, as integration and commmitment interact, with dropouts depending on commitment at the time of the decision. It remains to be seen whether the theory applies to this course, and whether its concepts can be usefully measured in practice. If they can, though, it may be possible to design remedies for weakness on any given aspect.
(My notes on Tinto's concepts and possible extensions, are available.)
Take measures of students' study habits: regularity, keeping up as the weeks go by, total time, devoting sufficient time to keep up, ...
Measure each student's inner interest in the subject. I.e. attempt to measure instrinsic motivation.
Perceptions by students of tutor quality: if tutors are a core part of the course, then variations in tutor quality may have a big effect. On the other hand, these perceptions may be caused by the students' success. Seeing whether the perceptions correlate with exam results on the one hand, and within groups on the other, would distinguish these possibilities.
Summary of kinds of measure
Self-estimate of understanding of the material.
Tutor's estimates of students' understanding
Prior learning of programming / computing science
Aptitude test
Motivation for and interest in the subject
Tinto variables of "integration"
Study skills / practices
Perceptions of tutor quality

Possible applications

Depending on what is found, there could be a number of different kinds of action taken in future:

Change selection methods for admission to the course
Identify weak students on the course and alert them, or concentrate teaching resources on them.
Identify weak and strong points in the course delivery, and modify that.

Feedback, questions

Any participant who would like feedback of any kind can obtain it from Rebecca Black (9906622b@student.gla.ac.uk).

Both Rebecca Black and Steve Draper will be happy to answer any questions about this.

Confidentiality

The identity of individual students is, and will be kept, confidential to the investigators, Rebecca Black and Steve Draper, and will be used only to link answers to other information gathered about a student at other times. The data will not be used in any way related to assessment, and will not be available to staff teaching the course except as overall totals. If anyone does not feel comfortable in answering any of the questions put to them, they may stop at any time.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [localed] [this page]
[Top of this page]