COVID-19: Remote assessment survey results

Deputy President (Welfare)
Friday 3 April 2020

Note: this blog post will display far better on a desktop browser

We sent out a survey and 6,940 of you responded. We've used this to advocate for a 'safety net' policy, and it wouldn't have been possible without so many responses - so, thank you.

On Sunday evening, we opened a survey on remote assessments during the COVID-19 pandemic. We wanted to find out how people felt about plans to move to remote time-restricted assessment, and about potential alternative options. In just 48 hours, we had 6,940 responses - about half of all taught students. Thank you to everyone who filled it in and helped share it. We've made a short report on the results of the survey, and have distributed as much of the raw results as we can (considering GDPR compliance) with the reps.

1. Difficulties and timezones

From what we've heard from students over the last few weeks, there were 3 key expected difficulties that students are concerned about. We asked respondents to indicate if they expected to have either of these difficulties up until the end of the term. 

Difficulty Percentage of students
Difficulty finding a quiet space to work 54.3
Having childcare or other caring responsibilities  10.2
Difficulty accessing technology needed to study and/or take exams 31.8
None of the above 31.0
Prefer not to say 4.8

Over 64% of students indicated they'd expect to have at least one of these issues, with over half saying they would have difficulty finding a quiet space to work and nearly a third with difficulty to access technology. Clearly, the move to remote working and the current global situation is affecting a large number of student's ability to study.

We also asked students to indicate what timezone they expect to be in during their exam period to get a sense of where everyone is located now, and how big the issue of students taking exams at silly times will be. The distribution of results is shown below. 

A majority of students (57%) are still in a London timezone, with 39% in a timezone ahead, and 4% in a timezone behind. The main clusters are in Western Europe, and Eastern Asia. There is a degree of good news here, as it seems not many students will need to be waking up extremely early to take exams (as this is mainly a worry for students on an earlier timezone). Departments are allowed to move their exam times by 5 hours, so hopefully students doing exams at silly hours can be mostly minimised.

2. Comfort with current form of assessment

We asked students to indicate, on a scale of 0-10 where 0 is 'Not at all comfortable' and 10 is 'Extremely comfortable', how comfortable they are with the current plan of remote assessment, and how comfortable they would be with the current plan plus a 'no detriment/safety net' policy. Such a policy would likely mean that, as long as a student has met the requirements necessary to pass, they will achieve at least the average mark they were on from all assessments up until the College closure due to COVID-19. Naturally, there would have to be exceptions where there isn't prior assessment to judge off.

The table below summarises the current comfort, the comfort with a safety net and the percentage of students who would feel at least or more comfortable with a safety net. It's split by department and also includes the percentage of students who indicated an expected difficulty.

Department Number of responses  % with expected difficulties Comfortable now avg. rating Comfortable w/ safety net avg. rating % as or more comfortable with safety net
Aeronautics 360 61% 4.67 7.58 90%
Bioengineering 437 63% 4.69 7.86 93%
Business School 422 72% 4.32 8.25 91%
Centre for Environmental Policy 14 71% 5.64 8.57 100%
Chemical Engineering 502 70% 4.59 7.54 92%
Chemistry 372 62% 4.97 7.99 93%
Civil and Environmental Engineering 300 68% 4.83 7.32 90%
Computing 388 51% 5.74 7.90 87%
Design Engineering 116 59% 5.28 7.72 91%
Earth Science and Engineering 279 66% 3.85 7.45 92%
Electrical and Electronic Engineering 361 58% 4.04 7.23 91%
Life Sciences 490 70% 4.25 7.47 92%
Materials 240 66% 4.78 7.79 88%
Mathematics 621 58% 4.75 6.98 88%
Mechanical Engineering 513 70% 4.75 7.69 91%
Medicine (MBBS) 561 61% 5.32 7.23 87%
Medicine (other) 204 69% 4.91 7.68 93%
Physics 754 73% 3.35 7.61 93%
Other 6 67% 4.17 7.50 83%
Grand Total 6940 65% 4.58 7.58 91%

On average across all departments, students reported a comfort level of 4.58 with the current plan which increased to 7.58 when considering a safety net policy. In every department (both with undergraduates and taught postgraduates), a vast majority of students would feel at least as comfortable (91%) and more comfortable (78%) with a no detriment/safety net policy. The average difference in comfort levels does vary across department: the largest increases in comfort came from Physics (4.3) and the Business School (3.9), and the smallest increases from Medicine MBBS (1.9) and Computing (2.2).

3. Disparities in comfort levels

We've compared the comfort levels of students who reported to have one of the difficulties listed in section 1, compared to those who didn't report an expectation to experience any of these difficulties - the results are summarised in the graph below. A large disparity in current comfort levels can be seen between those who didn't report any difficulties (5.9) and those who reported at least one (4.0), as well as those who preferred not to say (4.0 also). These all leveled out and the disparity nearly completely disappears when adding in a no detriment/safety net policy to 7.7 for those with neither difficulty, 7.5 for those who reported at least one and 7.3 for those who preferred not to say.

A similar, but less stark, effect is seen with students reporting specific learning difficulties (SpLD). With the current plan, those who reported to not have an SpLD felt a comfort level of 4.6 whereas those who reported one a level of 3.9 (and those who preferred not to say 3.6). With a no detriment/safety net policy, the disparity in these scores narrow significantly becoming 7.6, 7.5, and 7.2 respectively.

4. Alternative forms of assessment

We also asked students to indicate which alternatives to remote, time-restricted assessments they would prefer to the current plan (without a no detriment/safety net policy). Three options were given - chosen as a representative of other forms being used across the sector - and students could choose as many as they wish, and also write an option themselves if they wanted. The table below summarises the result.

Alternative % preferred
Replacing timed exams with coursework (a take home exam) that would be completed over a long (day or short number of days) period 58.3
Cancelling as many exams as possible (within external regulations), and progress/graduate students using alternative forms of assessment 35.1
Cancelling as many exams as possible (within external regulations), and progress/graduate students based on already ascertained information 37.5
None of these 16.6
Other (write in) 6.7

The option that more students would prefer was replacing timed exams with coursework - 58.3% indicated this would be a preferred option. 16.6% thought no other option was preferable, and 6.7% wrote in an alternative. Probably the most popular written in alternative was to postpone exams by a few weeks or until autumn.

Conclusions

What does it all mean? I think there are 4 key conclusions to draw:

1. A majority of students will experience difficulties that will impact their ability to study and/or take remote exams - this will likely lead to a vast number of  mitigating circumstance cases.

2. There is general discomfort with the current plan for assessment, and most students would prefer something different to happen.

3. The current form of assessment causes more discomfort for students facing certain difficulties compared to those who have said they won't face these.

4. There is a mixed response to what alternative assessments could be taken, so the implementation of a no detriment/safety net policy with the current form of assessment would be preferred by a greater number of students than any alternative form of assessment (78% are more comfortable with this policy whereas 58% would prefer the most popular alternative assessment).