Policy paper

Quantifying the UK Data Skill Gap - Summary version

Published 18 May 2021

Introduction

Data is now the driving force of the world’s modern economies. Driven by ever-increasing advancements and use of technology, never before have consumers and industry alike generated so much data.

As technology continues to progress, the demand for ‘data skills’ continues to rise dramatically. There are varying figures on how big the data economy is depending on how it is defined. The Department for Digital, Culture, Media & Sport (DCMS) estimates that in 2016 the value of the UK data economy was between £61.3 billion and £73.3 billion. It is expected that its value is considerably different now. However, the full economic potential of the data economy is yet to be realised. At full potential, researchers estimate that the UK data economy could have potentially been worth as much as £125 billion in 2016.

In hand with technological changes, changes in structures of employment have also contributed to this demand; jobs are moving from routine tasks towards complex expert thinking and communication.^{[footnote 1]} Building analytical capabilities will therefore be extremely important for UK businesses to remain competitive globally.^{[footnote 2]} As set out in the National Data Strategy, published in September 2020, the data revolution has implications not only for experts with advanced analytical skills, but also for the entire UK workforce. While not every worker needs to become a data scientist, everyone will need a basic level of data literacy to operate and thrive in increasingly ‘data-rich’ environments.

Furthermore, research has shown that effective utilisation of data can deliver significant productivity benefits to businesses and to the wider economy. Nesta research has proven that data- active companies are more productive than those that are more hesitant to embrace data.^{[footnote 3]} The ‘Big Data, Big Bang?’ report by the University of Brussels/ McKinsey & Co. finds that more productive firms are also faster adopters of big data than their industry peers.^{[footnote 4]}

With the increasing importance of developing data capabilities, employers are seeking professionals skilled in the architecture, retrieval, and analysis of the vast wealth of data available. DCMS commissioned research predicts that data analysis will be the fastest growing digital skills cluster, increasing by an estimated 33% in the next five years. It is therefore important for policy makers to understand the existing and future supply of skills in order to meet this demand in the UK. A considerable body of research has demonstrated that the UK suffers from a data skills gap. However, there is disagreement over its size, and whether this gap will grow as a result of the UK’s post-Brexit relationship with the EU.

In this context, DCMS has commissioned a quantitative assessment of the mismatch between the demand and supply of data skills, recommending regional and industry specific measures wherever possible.

Opinium Research was commissioned to:

• define the ‘data skills gap’ in order to be able to measure it
• measure and monitor the ‘data skills gap’ and identify the origin of the skills gap, the sectors most in demand for skilled workers and the current supply of skilled workers
• investigate drivers behind ‘data skills’ demand

This is a summary report highlighting key findings and statistics from this research. Please see the full report for further detailed information. The research looks to define the data skills gap in the UK. Our definition of data skills is any skill that involves the gathering, interpreting and communicating of data, often as part of a team.

We take a holistic approach, recognising that within individual businesses, skills needs are different and those needs are met not by single employees, but by teams working together. These skills cover a range of hard, technical skills and softer, generalist skills relating to good working practices, in the context of data.

To understand the data skills gap in the UK, we spoke, from end May to early June 2020, to 1,045 businesses across the UK, 5,000 UK workers and 1,000 students in higher education or training in the UK. This large-scale study of businesses, employees and students provides evidence on: (i) the overall need for data skills in the UK economy; (ii) the specific need for data skills, depending on business sizes and sectors; and (iii) where a future focus on training and recruitment might lie.

Recruitment freezes experienced in the early stages of the COVID-19 pandemic may have impacted the estimates of data skills demand incorporated in this report. It is, however, too early to understand the medium-term impact of the COVID-19 pandemic.

The findings of this summary report are based on both desk research and quantitative online surveys. The researchers would like to thank Danya Long and Sandy Grom at DCMS, Dr Matthew Forshaw from the Alan Turing Institute and Newcastle University, members of the Data Skills Taskforce and Professor Pat Tissington, Academic Director: Employability and Skills of Warwick University.

The demand for data skills

The scale of demand

Nearly half (48%) of businesses were recruiting for roles that required data skills.

When recruiting for data roles, businesses were most often looking for a “data analyst” (12% of businesses surveyed were recruiting for this role), especially at junior or entry level.

There are potentially 178,000-234,000 data roles to be filled.

Hard data skills^{[footnote 5]} are relevant to a range of workers, not just those that are in specialist data roles. We estimate on this broader level that UK businesses in need of hard data skills are recruiting for 215,000-234,000 roles. This range is based on 215,000 roles in businesses that require hard data skills beyond just basic IT skills,^{[footnote 6]} while a further 19,000 roles are in businesses that only require basic IT skills as a hard skill.

Our broad definition of ‘data skills’ covers a range of roles, from the more data-heavy specialist roles where employees are using data for the majority of their work, to more generalist roles requiring some hard data skills for a smaller proportion of their work. When narrowing the definition down to look at businesses recruiting for specific data specialist roles and those that require ‘hard’ data skills, we estimate UK businesses are recruiting for 178,000 vacancies. In these roles the majority of their work will centre around data and require more advanced data knowledge.

Almost half of businesses (46%) have struggled to recruit for roles that require data skills.

Over the last two years, just under half (46%) of businesses have struggled to recruit for roles that require data skills. The difficulty of recruitment differs by business size, although this is also the result of smaller businesses being less likely to recruit for data roles in the first place. Over half (55%) of large and medium (54%) size businesses have struggled with recruitment compared to 30% of smaller businesses and one in ten micro businesses.

Businesses tend to find it difficult to recruit for roles in higher demand. One in ten (9%) have difficulty filling a “data analyst” role, followed by 8% for a “Head of Data” position.

The skills we need

Most businesses think they have the skills they need.

Despite the fact that just under half of businesses are seeking to fill data roles, almost three fifths (58%) think that their organisation has sufficient data skills to meet their current and future needs, while 14% do not think they do, and over one in four (27%) are neutral on whether they do or not. Many companies may, however, believe they have the skills that they need due to a lack of understanding on how data skills may benefit them in the future.

Businesses need more information management skills, knowledge of emerging technologies and solutions, data communication skills, and database management skills.

Based on respondents’ views on importance and current levels of performance, we have identified the skills that companies most require. These are: information management, knowledge of emerging technologies and solutions, data communication skills, communication and database management. Overall, the skills cover a range of hard and soft skills.

Of the top three soft skills required by employers,^{[footnote 7]} the gap in communication and adaptability is more acute in medium-size businesses, whilst the gap for an “analytical mindset” was most strongly felt in small businesses.

For hard skills, the gap for information management and data communication is particularly acute in small businesses. This may be caused by small businesses showing a generally lower performance across all data skills, compared to other business sizes. In particular, small businesses show poorer performance in hard data skills compared to medium and large sized businesses.

Small businesses are significantly more likely than medium and large businesses to have no existing data roles, both at board level and at lower levels, with a third of small businesses having no existing data roles. They are also significantly less likely than medium and large businesses to be recruiting for any data roles.

Around a quarter of businesses said that their sectors had insufficient data skills in machine learning (25%), programming (22%), knowledge of emerging technologies and solutions (23%) and advanced statistics (22%).

The supply of data skills

This report considers two key sources of data skills supply. New talent primarily in the form of students coming from Higher Education and up-skilled workers. However, analysis of Higher Education Statistics Agency data prepared for the Data Skills Taskforce^{[footnote 8]} estimates that there is very limited scope, especially in the short run, for Higher Education to fill the gap alone. The estimated potential supply of data scientists from UK universities is unlikely to be more than 10,000 per year, based on data on graduates from UK universities between 2017-2018. It is therefore vital to look at upskilling of the current workforce to try and plug the gap in demand for data skills further.

Future skills provision

Employers are positive about their ability to identify and source training needs, often because it is happening internally.

Employers express positive attitudes to data skills training. Two thirds of employers report they are confident in identifying skills gaps within their workforce (64%), while a slightly smaller proportion of employers indicate they are confident in where to find resources to train their employees (57%).

Training is most likely to be taking place internally. Over half (56%) of businesses surveyed said that their organisation preferred to develop data skills internally rather than outsource or recruit new workers with those skills. Despite this apparent preference for internal training, UK businesses are still recruiting for up to 234,000 data roles.

In our survey, just under a third (29%) of employers highlighted cost as one of the biggest barriers to data skills training. This is followed by difficulties in finding training that is relevant to an organisation and has practical relevance to an individual’s day-to-day job. Universities were not seen as a significant source of data skills training for staff. Only 14% of businesses who have sent employees to data skills training in the last two years have put them through university taught programmes, and 7% have sent their employees to university continuing professional development courses.

Interestingly, smaller businesses (49% of micro businesses vs 23% of large businesses) are more likely to say that there are no barriers, despite or perhaps because they are less likely to access data training for their employees.

Three in five (60%) workers who have not received any data skills training said data skills do not add any significant value, compared to 19% of those who have received this type of training. This suggests that those that do receive training may be significantly more likely to see the value of data skills in their job.

Workers in smaller businesses are also less likely to say that data skills add significant value to their job or company. Two thirds (68%) of sole traders and half (48%) of those working in micro businesses (2-9 employees) said there was no significant impact, compared to over a third (35%) of those in medium sized businesses (50-249 employees), and 36% in large businesses (250+ employees).

Cost and time constraints were the biggest barriers to accessing data skills training

For workers, cost (29%) and lack of time (28%) were the most common barriers cited to accessing data skills training, followed by a lack of availability of the kind of training they are interested in (13%).

Data skills training is felt to improve efficiency and productivity and reduce stress, but there isn’t enough

Workers told us when training is relevant to their day-to-day job, it improved their productivity or innovation and reduced stress levels. It is also apparent that the need for data skills is increasing, and that the majority of workers are interested in acquiring these skills. Seven in ten (70%) workers interviewed expressed interest in seeking out data skills training, and half (46%) stated that the need for them to have data skills has increased over the past five years (in contrast, only 4% said the need has decreased). In spite of this, we found that across all employees, half had not received any data skills training within the last two years. The lack of training despite high demand from both employers and employees suggests that the challenge of ensuring appropriate training provision is complex.

The current priorities for businesses for specific hard data skills training are data ethics and basic IT skills. These two skills are also in the top five data skills that employees currently receive training for, along with leadership, project management and industry expertise. For soft skills, communication, professionalism and problem solving are the priorities.

The next generation

There is a mismatch between students’ evaluation of their industry or subject matter expertise and the reality. Businesses want them to develop leadership and communication skills.

Around a quarter of businesses said that graduates who work with data need to develop their leadership (26%) and communication (23%) skills.

Students (classified as any adults in higher education or training) are generally optimistic about their current skills and expect to improve further. For example, over two fifths (45%) of students rate their current industry/sector expertise as excellent/good, while over two thirds (68%) expect it to be excellent/good at the end of their studies. More than half expect to be good or excellent at advanced statistics (61%), data visualisation (69%), machine learning (59%) and knowledge of emerging technologies (68%).

Half (49%) of those in higher education or training felt that the training or education path to become a data scientist is clear, while 39% thought it was unclear.

Of those students who said the path to become a data scientist is unclear to them or they were not sure, two in five (39%) haven’t sought advice to guide them on this, while three in five (61%) have. The places students most commonly have sought guidance include the internet (39%), speaking to careers advisors in university (15%), with the same proportion speaking to university staff. This shows that even seeking guidance has not helped many students and suggests that there is much more that could be done to improve existing guidance.

215,000–234,000

The number of roles that require the hard data skills that UK businesses are seeking.

178,000

The number of specialist data roles that require technical or specialist knowledge that UK businesses are seeking.

Workers told us when training is relevant to their day-to-day job, it improved their productivity or innovation and reduced stress levels. Despite the strong interest from workers and the increasing need for data skills, half (50%) had not received any data skills training within the last two years.

Just under half (49%) of those in higher education or training felt that the training or education path to becoming a data scientist is clear, while just under two fifths (39%) thought it was unclear.

Top 10 roles organisations are currently recruiting for:

1. Data Analyst

2. Head of Data

3. Data Manager

4. Chief Technology Officer

5. Data Protection Officer

6. Chief Information Officer

7. Data Scientist

8. Data Technician

9. Chief Data Officer

10. Data Engineer

Top 10 skills that companies need to improve (based on a comparison of importance and current performance):

1. Information management

2. Knowledge of emerging technologies and solutions

3. Data communication skills

4. Communication

5. Database management

6. Data literacy

7. Data ethics

8. Analysis skills

9. Analytical mindset

10. Adaptability

Figure 1. Percentage of UK businesses who have struggled to recruit for a data role over the last two years – by business size

• All businesses	46%
  Micro	10%
  Small	30%
  Medium	54%
  Large	55%

Figure 2. Data skills gap by percentage points – by region

All employers	14%
East Midlands	12%
London	12%
North East*	22%
Northern Ireland*	8%
North West	13%
West Midlands	15%
South East	18%
Wales*	12%
Yorkshire and the Humber	19%
East of England	1%
South West	12%
Scotland	9%

*low base size, differences not significant

Figure 3. Data skills gap by percentage points – by industry

All employers	14%
Banking	11%
Computers*	19%
Education	19%
Manufacturing	16%
Retail	7%

*Computers includes those working in computer hardware, computer software and computer services

Figure 4. Data skills gap by percentage points – by business size*

All employers	14%
Micro*	-1%
Small	17%
Medium	19%
Large	14%

*The negative score for micro businesses is because they are less likely to be recruiting for data roles in the first place

Top 10 skills businesses say their sector has insufficient skills in

1. Machine learning - 28%

2. Programming - 24%

3. Knowledge of emerging technologies and solutions - 24%

4. Advanced statistics - 24%

5. Data visualisation - 23%

6. Storytelling - 23%

7. Analytical mindset - 22%

8. Creativity - 22%

9. Data ethics - 21%

10. Critical thinking - 21%

Top 10 skills that businesses say graduates are lacking

1. Basic IT skills - 18%

2. Data ethics - 17%

3. Machine learning - 16%

4. Programming - 15%

5. Data processing - 15%

6. Data communication skills - 14%

7. Knowledge of emerging technologies and solutions - 13%

8. Advanced statistics - 13%

9. Analysis skills - 13%

10. Data visualisation - 13%

Top 5 priority skills businesses want to improve in the context of working with data:

1. Communication

2. Professionalism

3. Problem-solving

4. Data ethics

5. Basic IT skills

Top 5 data skills that employees currently receive training for:

1. Leadership

2. Basic IT skills

3. Project management

4. Industry / sector expertise

5. Data ethics

Conclusions/observations

This research has identified significant demand for data skills in the UK workplace.

However, the supply of data skills from universities is unlikely to meet this demand, particularly in the short term.

There are potentially 178,000-234,000 data roles to be filled. Yet, internal analysis of Higher Education Statistics Agency data prepared for the Data Skills Taskforce estimates the potential supply of data scientists from UK universities is unlikely to be more than 10,000 per year, based on data on graduates from UK universities between 2017-2018.

This difficulty of filling the data skills demand gap is also corroborated by many companies telling us that they have found it challenging to fill data roles in the past two years, with financial costs and lack of the right combination of skills being the topmost barriers. There also appears to be a knowledge/communication gap issue that could pose a barrier to fulfilling this demand, with many students telling us that they do not feel there is clarity on the path to becoming a data scientist even when they have sought guidance.

Upskilling the workforce will therefore be a vital step to bridging the UK data skills gap.

Strikingly, half of all workers surveyed reported no data skills training within the last two years, and sizeable proportions reported a complete lack of interest. Universities were not seen as a significant source of training by businesses; only 14% of businesses who have sent employees to data skills training in the last two years have put them through university taught programmes, and only 7% have sent their employees to continuing professional development courses at a university. Most of the internal training is happening either in-house, via private training companies or through online programmes (e.g. webinars).

Education and training providers have a critical role to play in shaping curricula that looks to address the skill gaps identified in this report and to offer a diversity of training options that also cater to those at lower price points.

There is further discussion of the data skills gap in the full report. This includes setting out measures being taken to address the UK data skills gap.

1. OECD (2019). Skills Matter: Additional Results from the Survey of Adult Skills, OECD Skills Studies, OECD Publishing, Paris. ↩

2. SAS/The Tech Partnership (2014). Big Data Analytics: Assessment of Demand for Labour and Skills, 2013-2020. ↩

3. Nesta (2015). Skills of the Datavores: Talent and the Data Revolution ↩

4. University of Brussels & McKinsey&Co (2016). Big Data, Big Bang? ↩

5. Hard data skills include: Basic IT skills, Information management, analysis skills, data ethics, programming, database management, data processing, data literacy, data visualisation, advanced statistics, machine learning, data communication skills, knowledge of emerging technologies and solutions. Information on why this list was compiled can be found in the full report. ↩

6. Basic IT skills as part of a wider set of skills required to analyse data. ↩

7. Soft data skills include: professionalism, communication, problem solving, collaboration, adaptability, critical thinking, subject matter expertise, industry/sector expertise, analytical mindset, creativity, project management, leadership, curiosity and storytelling. Information on why this list was compiled can be found in the full report. ↩

8. The Data Skills Taskforce was set up – partly in response to a recommendation in Nesta’s Analytic Britain report – to act as a knowledge and best practice-sharing forum across key participants from industry and higher education, and to promote data skills and analytics. It has a wide range of members from industry, academia, Royal Societies and government. ↩