The Department for Education has published independent
research summarising users' and experts' views on the
'Employer skills' and 'Employer perspectives'
surveys.
The report assesses whether data collected from the
surveys is useful for answering policy and research
questions. A total of 27 different institutions
contributed to the research, representing a range of
sectors and interests.
Conclusions
London Economics were tasked with undertaking a
consultation of ESS and EPS users to assess whether the
survey data collected was sufficient to answer key policy
and research questions relating to employer demand for
skills. We were also asked to gather information on
whether the current data collection approach and surveys’
structure could be achieved in a more integrated,
cost-effective manner. The consultees represented a range
of different sectors and interests, including government,
the private sector and the academic community.
Impact
The consultees expressed interest in a wide range of key
policy and research questions relating to employer demand
for skills, demonstrating the diversity of need among ESS
and EPS users – and indicating the challenge in meeting
these many diverse needs through just two survey tools.
The most common theme was the use of the surveys to
identify skills shortages, gaps and mismatches, and many
consultees mentioned the ability of the surveys to
generate this evidence consistently and robustly over
time and at more refined geographic levels, as particular
merits.
Consultees described the impact of the surveys on
national level policy, as well as in developing regional
and local skills strategies (and for informing funding
decisions). While some consultees highlighted the role of
the surveys in facilitating training providers to target
skills gaps, others questioned whether the surveys
collected information in sufficient detail for this
purpose. As such, although there was significant impact
identified, there was a belief that awareness of the
surveys and their associated impact could be
increased.
Options for cost savings
Cutting questions or modules
Many consultees suggested changes to existing questions
and/or additions of questions to the ESS and EPS surveys
in order to enhance the evidence collected on topics
relevant to their work. Very few suggested cutting
existing questions; however, a number did question the
usefulness of certain sections. There was no consensus on
the questions or topics that could be straightforwardly
cut. It was difficult to identify any topics or questions
within the questionnaire marked as redundant by one
consultee, which were not used by at least one other
consultee.
Reducing sample sizes
ESS and EPS users were also generally satisfied with
current sample sizes, with some identifying the large
sample size of the ESS as being ‘the main value of the
survey’. When asked about a possible reduction in the
sample size of the ESS, many consultees expressed strong
views against this change, as it would jeopardise their
ability to produce evidence at sub-regional (or lower)
levels.
Merging data sets
Many consultees saw merit in a potential merger of the
ESS and EPS into a unique questionnaire covering all
topics relating to employer demand for skills. While
merging the surveys has some benefits in terms of
possible cost savings, there would be a number of
non-trivial methodological issues that would need to be
addressed before any attempt to merge the surveys took
place. Addressing these methodological issues would
probably significantly reduce any savings that might be
associated with merging, but failure to do so would
undoubtedly jeopardise a well-regarded suite of surveys.
Given the fact that the surveys are fundamentally
different (for instance in the nature of the questions
asked within the same topic), merging the two data sets
offers limited gains.
Survey frequency
In terms of survey frequency, the current biennial data
collection of the ESS and EPS seems to suit a majority of
survey users adequately, and offers an acceptable
trade-off between cost, coverage and timeliness.
Sampling frame
Significant benefits could be achieved for some users,
especially those in government departments and the
academic community, if the surveys were sampled from the
IDBR instead of from Experian and were thus directly
ready for analyses in combination with other ONS sources.
Although the IDBR would facilitate data linking, the
ESS/EPS data may not become fully comparable to other ONS
data sources due to other sampling differences (for
instance, the skills surveys being collected at
establishment level, while most other data is available
at enterprise level). However, given the benefits
associated with this option, even for a small number of
users, it might be worth considering what the indicative
costs of this option might be.
Currently, all establishments with at least two workers
are eligible for the surveys. If this sampling frame were
decreased to include only establishments with at least 5
workers, some significant cost savings could be achieved
in the fieldwork and through a potentially smaller
required overall sample size. However, such savings would
come at the expense of less representative surveys and
discontinued time series for small businesses.
Survey administration
Some efficiency gains could potentially be achieved from
commissioning the ESS and EPS together – and over a
longer timeframe. Compared to the current approach where
each round of surveys is commissioned one at a time, 41
commissioning two or three waves of the ESS and/or the
EPS would result in increased institutional knowledge
within the contractor’s organisation, as well as a
reduction in the ongoing burden of resubmitting large
research proposals. It would be expected that the
benefits associated with increased certainty and reduced
burden on the contractor side would be shared with the
Department and present themselves as (not insignificant
cost savings). However, comprehensive monitoring and
quality assurance processes would need to be in place to
ensure value for money for the taxpayer.
Cost savings versus risk
Although there were several suggestions put forward in
relation to the future administration/ composition of the
surveys that might deliver cost savings, many of these
would have a detrimental impact on the usefulness of the
survey for other users; result in an increase in costs in
the short term; or generate a risk in relation to the
year-on-year quality and comparability of the data. As
such, with the exception of the possibility of procuring
the administration of the surveys on a longer-term basis,
where some significant savings might be generated without
any impact on the surveys themselves, the proposed
options appeared to introduce change (and costs) without
necessarily delivering any benefits that would warrant a
fundamental re-organisation of the surveys.
Future use and benefits
Overall, a majority of consultees expressed a wish to see
the surveys continue beyond the closure of UKCES. As
evidence of the reputation of the surveys, consultees
pointed to the breadth of users, the surveys’
international status, and that fact the surveys have
inspired other international data collection exercises.
However, a number of consultees expressed the opinion
that awareness and use of the surveys could be higher,
given their significant value. In light of the challenge
to make cost savings, evidence from the consultation
exercise suggests that the focus of policy makers
involved in the management of the surveys should be to
increase the awareness and use (and impact) of the
surveys. A concerted effort to reduce costs through
amending the methodology or content of the surveys would
likely be self-defeating, and would result in a suite of
surveys that has limited use. Given the fundamental risks
that the UK economy now faces, the continuation of
well-respected, reliable, authoritative and transparent
data sources on the employer demand for skills is a
(currently unrealised) benefit in its own right and
should not be put at risk.
