NFER publishes National Tutoring Programme 2020/21 evaluation findings

NFER and its partners have completed their independent evaluation of the first year (2020/21) of the National Tutoring Programme (NTP) Tuition Partners (TP) pillar and their impact evaluation of the Academic Mentoring (AM) pillar.

The delivery and evaluation of both pillars took place under extreme and unprecedented circumstances due to the disruption caused by the Covid-19 pandemic. There were different assessment arrangements for Year 11s (teacher assessed grades) and Key Stage 2 assessments (for Year 6 pupils) were cancelled. Findings should be interpreted in this context and with caution.

The evaluation of the Tuition Partners programme found that higher amounts of tutoring were related to better scores in English in primary schools, and with better grades for Year 11s in maths and English (teacher assessed grades/TAGs). In schools with higher proportions of pupil premium pupils taking part, the Tuition Partners programme had a positive impact on Year 11 grades (TAGs) for those pupils. 

NFER’s Head of Classroom Practice and Workforce, Ben Styles, said: “This evaluation was carefully designed to provide schools with evidence to guide decisions and support disadvantaged pupils, but a rapidly changing situation at the height of the pandemic presented numerous methodological challenges, meaning findings should be interpreted with caution.

“The limited reach of the programme across pupil premium pupils in participating schools as a whole meant benefits were difficult to detect across pupil premium pupils, but tuition was associated with improvements in English performance for primary schools and both English and maths among Year 11 pupils.

“As schools now receive funding to spend on whichever model of tutoring they choose, it is vital that the research community provides them with the evidence they need to guide decisions. The best way to generate such evidence is a programme of randomised controlled trials designed to optimise what is, at present, a hugely diverse market.”

In addition to changes to assessment arrangements, the first year of NTP’s delivery and evaluation needed to take account of:

• Partial school closures in January 2021 which affected the intended delivery models. Many schools paused Tuition Partners tuition or chose to wait to start until after schools re-opened, shifting delivery to later in the academic year. More tuition partners sessions took place online than originally expected. For Academic Mentoring, Covid-related absences and partial school closures meant some re-deployment of mentors by some schools. 

• A relatively small proportion of pupil premium (PP) pupils in participating schools were selected for tutoring, meaning a large number of pupil premium eligible pupils were included in the analysis that did not receive tutoring. The evaluation aimed to explore the effect of the programme on pupils eligible for pupil premium funding in participating schools compared with similar pupils in schools that did not participate. But this development made comparison (and therefore analysis and interpretation) challenging and less reliable. 

Independent evaluation findings:  

Tuition Partners 

While it was challenging to detect an impact across all pupil premium pupils in schools that accessed tuition through Tuition Partners, the evaluation found that:

• Higher amounts of tutoring were related to better assessment scores in English in primary schools.   
• Higher amounts of tutoring were also associated with better teacher assessed grades for Year 11s in maths and English. 
• In schools with a higher proportion of pupil premium-eligible pupils taking part in tutoring through Tuition Partners , the programme had a positive impact on Year 11 Teacher Assessed Grades (TAGs) in English and in maths. This analysis was based on a smaller sample of schools with some different characteristics to all tuition partner schools, and so may not be generalisable to all tuition partner schools.
• Most schools were satisfied with the quality of tuition (80 per cent of school leads were either ‘very’ or ‘somewhat’ satisfied).  
• 232,892 unique pupils were enrolled onto the tuition partner programme with just under half (46%) eligible for pupil premium funding (of those for whom data was provided).
• 6,082 schools signed up to receive tutoring through the tuition partners pillar, and over half of these signed up schools (59%) had 24% or more pupil premium pupils (the reporting category for tuition partners).

Academic Mentors (AM) 

In our analysis looking at Year 11 PP-eligible pupils in AM participating schools, the majority of PP-eligible pupils did not receive academic mentoring. Whilst the results indicate a positive effect of +1 additional months progress in maths in Year 11 compared to pupil premium-eligible pupils in Year 11 in comparison schools, there is a degree of uncertainty around this result. The large proportion of analysed pupils who did not receive academic mentoring, means it is unlikely that any difference seen in this analysis model was due to academic mentoring. This means that this evaluation is unable to conclude, with any certainty, whether or not academic mentoring had an impact on the English and maths attainment outcomes of those pupils who received it.  

NFER recommends: 

• Future programmes should either clearly define for whom they are designed, or acknowledge that schools may have different views about which of their pupils most need and would benefit from tuition.  
• More should be done to target support at disadvantaged pupils. This recommendation is based on a) this evaluation’s findings when restricting to the higher pupil premium subsample, b) previous research from NFER and other organisations which shows the disadvantage gap remains wide and c) evidence in the EEF toolkit about targeted tuition.
• Schools need greater clarity about the expectations of their role in managing and delivering (different kinds of) tuition and, where necessary, additional support for doing so. 

• Schools and tutors need to work together on how best to ensure tutoring is aligned with (and additional to) classroom teaching, tailored to pupils’ needs, and that pupils complete their tuition.

• A programme of evaluation should be developed to help explore which models of tutoring are most effective for which pupils and in what circumstances. This could evaluate different types of tutoring through a range of Randomised Controlled Trials (RCTs).

The evaluation was designed and carried out by a consortium, led by NFER, with Kantar Public and the University of Westminster, and commissioned by the Education Endowment Foundation (EEF).