This report from the University of Warwick’s Centre for Competitive Advantage in the Global Economy (CAGE), draws on unique new research looking at the long-term consequences of poor performance in high stakes exams due to random factors not linked to ability.
- Examination candidates can have a bad day for many reasons unrelated to knowledge, skill, or cognitive ability. Possible causes include minor infection, migraine, hay fever, menstruation, disturbed sleep, and atmospheric pollution. When the stakes are high, as with exams used to rank students or admit them to further training or employment, there can be permanent consequences for the individual and for society.
- There is a lack of evidence on these consequences. Empirical challenges include the difficulty in identifying the return to cognitive ability separately from the return to doing well on the examination.
- Our solution to this problem is to examine the consequences of fluctuations in a random factor on exam performance. We use fluctuations in air pollution for this purpose. When the same student takes multiple exams, and exposure to ambient pollution varies randomly from day to day, we can use the associated variation in performance to measure the component of a student’s score which is related entirely to luck. The results show that transitory ambient pollution exposure is associated with a significant decline in student performance.
- We then examine these students during adulthood (8-10 years after the exams) and we find that pollution exposure during exams causes lasting damage to post-secondary educational attainment and earnings later in life.
- Our analysis highlights that high-stakes exams provide measures of student quality that may be imprecise and misleading. These measures can lead to allocative inefficiency because students who have had a bad day because of factors outside their control are mis-ranked. After that, they are inefficiently assigned to further training and to different occupations and this reduces labour productivity overall.
- As well as illustrating these problems with high-stakes exams, our findings also expand understanding of the costs of pollution. They imply that a narrow focus on traditional health outcomes, such as hospitalisation and increased mortality, is not enough. The full cost of pollution includes loss of mental acuity, which is essential to productivity in most professions. The use of high-stakes exams to measure mental acuity then multiplies this loss over many years.