Independent report

Safer technology, safer users: The UK as a world-leader in Safety Tech (May 2020)

Updated 15 June 2023

Ministerial Foreword

The government is committed to ensuring that the UK is the safest place in the world to go online. It is essential that all users have the best possible protection against online harms, and that illegal content is identified and removed.

The government’s Online Harms White Paper set out a programme of action to tackle content or activity that harms individual users, particularly children, or threatens our way of life in the UK - either by undermining national security, or by eroding our shared rights, responsibilities and opportunities to foster integration.

I am delighted, therefore, to introduce this landmark Safety Tech sector study – as we committed to do in our response to the online harms consultation. The UK is a global leader in improving online safety, and is advancing shared efforts across the world. As a pioneer in emerging technologies and innovative regulation, the UK is well placed to seize the opportunities offered by new and emerging technologies across the online safety sector.

Already, the UK has made significant technical breakthroughs and driven real innovation in advancing the effectiveness of content moderation, filtering and digital forensics.

The businesses and organisations showcased in this report are an essential part of ensuring the UK continues to be a world leader in the international fight against illegal and harmful content online.

The many Safety Tech start-ups highlighted in this report - including hubs in Edinburgh and Leeds, as well as London - show how the UK is at the forefront of cutting edge Safety Tech and is developing leading safety products that are already being used worldwide.

There is clearly enormous potential across the Safety Tech sector – both to keep internet users safe, and to foster the development of sustainable, high-tech companies across the country. The UK believes that technology itself is part of the solution to protecting users online. This report sets out our proposed measures to boost the Safety Tech sector in the UK, as well as measures to help users manage their own safety online and the wide range of Safety Tech products and services that already help make online platforms safer.

I am excited to see a thriving Safety Tech market playing a significant role in protecting users, whilst also driving digital growth. The strong desire across this industry to come together, speak with one voice and collaborate across key issues is both clearly evident and impressive. I look forward to seeing how this sector continues to grow from strength to strength into the future.

Caroline Dinenage MP

Minister of State for Digital and Culture

Executive Summary

This report provides an overview of the UK’s Safety Technology (“Safety Tech”) sector, including its market growth and potential. It highlights some of the UK’s most innovative businesses focused on tackling online harms through a range of technical solutions.

It is based on quantitative market analysis, and more than 50 interviews with leading industry stakeholders, government and policy leads, law enforcement, academia, think-tanks and charities.

Safety Tech providers develop technology or solutions to facilitate safer online experiences, and protect users from harmful content, contact or conduct.

The online world offers many benefits, but at the same time it can expose users to harm. Over the past decade a growing number of organisations have developed technology to help manage and reduce these risks, with the shared aim of keeping users safer online.

These organisations often:

• Work closely with law enforcement, to help trace, locate and facilitate the removal of illegal content online
• Work with social media, gaming, and content providers to identify harmful behaviour within their platforms
• Monitor, detect and share online harm threats with industry and law enforcement in real-time
• Develop trusted online platforms that are age-appropriate and provide parental reassurance for when children are online
• Verify and assure the age of users
• Actively identify and respond to instances of online harm, bullying, harassment and abuse
• Filter, block and flag harmful content at a network or device level
• Detect and disrupt false, misleading or harmful narratives
• Advise and support a community of moderators to identify and remove harmful content

The technology and services that these organisations provide are valuable to government and society. They help law enforcement and platforms to identify and stop some of the most serious illegal content such as child sexual exploitation and abuse (CSEA) ^{[footnote 1]}, and terrorist imagery and material. The technologies and services work to reduce the risk of users, including children, being exposed to harmful content, contact or conduct such as grooming, bullying, radicalisation and/or viewing self-harm material. They also help to tackle disinformation, and false or misleading narratives.

Safety Tech providers combine profit with purpose. Providers have been able to tap into the UK’s depths of expertise in fields such as artificial intelligence (AI), cyber security, RegTech and Tech for Good, and channel these into creating solutions that make a real difference to users online. As one consultee within this research noted:

“The greater the revenue we generate, the further we can invest in enhancing our platform, and the more people we will protect from harm.” UK Digital Forensics SME

Market Profile

Rapid growth in recent years (35% per annum), and positive expectations for enhanced growth to follow.

There are at least 70 dedicated Safety Tech businesses within the UK, as well as a wider community of over 100 organisations committed to protecting users online.

The Safety Tech sector has grown rapidly with an estimated 35% annual growth rate since 2016.

In 2019, the sector generated an estimated £226m in annual revenues.

We estimate that Safety Tech revenues in the UK could exceed £1bn by the mid-2020s assuming a comparable or higher growth trajectory in future years.

Some companies have seen even higher rates of growth. Some of the most established UK Safety Tech companies have grown at up to 90% per annum. It is anticipated that domestic and international demand will continue to grow rapidly in the years ahead.

High investor confidence

In addition to strong revenue growth, there is clear evidence of investor confidence in UK Safety Tech providers.

The past four years have seen external investment in the sector increase more than 800%, to a record year in 2019 with £51m raised across 19 deals, as the scale and maturity of companies within the sector has developed.

This investor confidence is reflected by most recent post-money valuation data for UK Safety Tech companies, which we estimate is valued (2018/19) at £503m

Detailed market consultations have flagged that given the significant growth of the sector, several of these firm-level valuations are likely to be considerably higher as of early 2020.

Potential emergence of UK Safety Tech Unicorns by the mid-2020s

Throughout the consultations undertaken when compiling this report there was a sector consensus that the UK is likely to see its first Safety Tech unicorn (i.e. a company worth over $1bn) emerge in the coming years, with three other companies also demonstrating the potential to hit unicorn status within the early 2020s. Unicorns reflect their namesake – they are incredibly rare, and the UK has to date created 77 unicorn businesses across all sectors (as of Q4 2019).

The potential to create one or more unicorns in the years to come demonstrates the significant high potential of the Safety Tech market as understood by the investment community – and the UK’s role as world-leaders in Safety Tech, pitching for the best balance between technology, ethics, and purpose.

“Good technology is exportable. The UK should want to protect children across the world.” - Large multinational Safety Tech provider

“We’ve seen a real change since 2016 – particularly among the gaming industry, that has moved away from ‘why would you want to filter’ to realising that social interaction matters to their platforms.” - Content Moderation Provider

A strong multidisciplinary talent base dedicated to online safety and technology in the UK

The UK’s Safety Tech sector includes a range of talent, and includes expertise in fields such as: data science; artificial intelligence and machine learning; the domains of language and semantics, policy and ethics; the disciplines of computer science, software engineering, computer information technology and systems; and also criminology, law, and the behavioural and social sciences.

The 70 businesses identified hire over 1,700 full-time equivalent staff in the UK, with emerging regional centres of excellence in London, Leeds, Cambridge and Edinburgh.

Outward-looking, socially-driven and international in scope

As a sector, UK Safety Tech providers are outward-looking, and recognise that issues relating to online harms cannot be tackled within one single jurisdiction, but rather require international collaboration and outreach. Export markets are growing - there is a keen desire to export and work internationally, in full recognition that technologies and solutions developed within the UK can have global relevance and application.

Despite the sector’s recent emergence, 47% of UK Safety Tech companies already have an identifiable international presence.

We also estimate that the UK providers are likely to represent at least 25% of international market share for independent Safety Tech providers.

UK Safety Tech as a Global Leader

The UK is recognised as a global leader in Safety Tech and its approach to online safety. International stakeholders particularly noted that the UK has been able to ‘merge the technical and ethical leadership’ required for being a leader in this space.

“The UK is the lighthouse for online safety. We’ve seen some of the first major steps taken by a country to regulate in the area of online harms.” - Large multinational safety tech provider

“There’s a lot of companies doing interesting things in the UK, and there’s huge market demand for solutions to these problems.” - UK content moderation provider

Recommendations

As part of this research we have sought to understand how the UK Safety Tech sector can be best supported to achieve further growth, and to help protect as many users as possible.

Theme	Detail
1. Promote and increase awareness of the UK Safety Tech sector	UK Safety Tech has a range of technical solutions to create safer online communities. However, demand-side awareness of these products is limited. Consultees highlighted the need for a sustained series of initiatives to promote and enhance awareness of Safety Tech among social media platforms, ISPs, schools, parents, governments and investors.
2. Support Safety Tech firms to access the right forms of capital for growth	Investor confidence within UK Safety Tech has clearly grown in recent years; however, the volume and scale of investment could grow further in the years ahead. Several stakeholders highlighted the value of grants or match funding to help them undertake exploratory projects, and the value of a competitive tax regime and support. Public procurement was also cited as a critical component of helping the sector to grow, with respect to both ease of access and increased opportunities to work with the public sector and law enforcement.
3. Getting the policy landscape right – with government providing leadership, guidance and appropriate legislation to address online harms	The UK Safety Tech sector perceives that there has been considerable policy activity in recent months, and there is a real need for this momentum to continue. This can be supported through enhanced consideration of ‘Safety by Design’ within regulatory development; and the development of strategy to promote cross-sector engagement, to set and enforce standards, and to help develop trust in Safety Tech products & providers.
4. Enabling improved access to data, including promotion of data-sharing, privacy by design, and collaborative partnerships	Artificial Intelligence (AI) tools play an essential role in helping companies detect and respond to online harms at scale. The effectiveness of these systems rely on the quality of the data on which they are built and tested. However, many consultees have described their difficulties in accessing high-quality data illustrating online harms. Current standards do not support interoperability, and mechanisms for safe and secure data-sharing are scarce. This is limiting the growth of a competitive market in AI-driven Safety Tech solutions.
5. Supporting innovation, cross-disciplinary research and development in Safety Tech	Consultees noted that they would welcome research councils (e.g. UKRI) establishing a bespoke research agenda for Safety Tech in the UK. There is a distinct need to establish ‘Safety Tech’ research opportunities within the UK, rather than include this as a cyber security component.

The primary recommendations (to government) from those we spoke to in compiling this report are:

1. Establish a UK Safety Tech Industry Association ^{[footnote 4]} to help create a consistent voice for the sector, and coordinate awareness-raising activity;
2. Raise awareness of Safety Tech. Work across sectors to showcase UK Safety Tech providers and create stronger networks to connect buyers and suppliers, and to drive adoption;
3. Grow safety tech exports, and work with the Department for International Trade (DIT) to review market opportunities and carry out trade missions.
4. Review the financial and practical support given to UK Safety Tech organisations, to ensure they have sufficient access to investment. As part of this, investigate the use of grants and challenge funds to drive start-ups, scale-up and innovation, and opportunities to commercialise work coming from the academic sector;
5. Update public procurement guidelines, to ensure that public sector organisations are making sufficient use of Safety Tech to protect their own systems.
6. Explore the opportunity for the UK Safety Tech market to address regulatory compliance needs, and support continued innovation;
7. Design and implement a sector-wide Safety Tech strategy, including mechanisms to help cross-sector collaboration - for example, regular government and industry roundtables or innovation networks;
8. Promote development and use of consistent and robust standards: The UK has already developed best practice standards in online safety. These should be promoted wherever possible, and new standards developed where appropriate.
9. Map the online harms data landscape and prototype solutions that could promote interoperability and data-sharing. As part of this, review current access routes to relevant data e.g. use of sandbox environments, and engagement with existing Government datasets.
10. Explore opportunities for enhanced academic research in UK Safety Tech with UKRI. Specific research calls should explore technical (and other) approaches to addressing online harms, including testing products in the marketplace, and evaluation of ‘what works’.

1. Introduction

1.1 Introduction

Perspective Economics, with advisory input from Professor Julia Davidson and Professor Mary Aiken (University of East London), has been commissioned by the Department for Digital, Culture, Media & Sport (DCMS) to conduct a sectoral analysis of the UK Online Safety Technology sector (referred hereafter as Safety Tech).

This report outlines the market profile of the Safety Tech sector in the UK. It focuses on establishing a common definition of the components of the sector, including the development of a market taxonomy, and understanding the range of Safety Tech related products and services that are available within the market.

This is the first piece of research that seeks to define and measure the nature, contribution and potential of this sector. We recognise that this is an emerging field, and welcome further discussion and comments regarding the findings and methodology.

1.2 Team and Acknowledgements

The study team included Sam Donaldson (Study Lead, Perspective Economics), Professor Julia Davidson (University of East London), and Professor Mary Aiken (University of East London). DCMS and Perspective Economics would like to acknowledge the consultees, nationally and internationally, who contributed to the development of this report through participation within workshops and one-to-one consultations with the research team. In total, our research engaged with more than 50 industry, academic, policy, and charitable representatives. This research would not have been possible without the several hundreds of hours contributed by stakeholders throughout October 2019 to April 2020.

Beyond the statistics set out within this report, the team were impressed by the dedication, commitment and vision of many of the businesses, charities, academics, and policy-makers consulted throughout this research. It is considered likely that Safety Tech in the UK will continue its rapid growth, and develop and implement highly effective solutions to keep the nation safer from illegal and harmful content and behaviours online.

1.3 Scope

This research seeks to identify providers of Safety Tech products or services, with a clear presence in the UK market (UK registered), and that are active and undertake commercial activity.

For research purposes, the following is considered within this report; however, we recognise the wider contribution of many organisations involved within the wider online safety ecosystem.

Research Scope

Safety Tech providers, which:

• have a clear presence in the UK market (registered and active status)
• demonstrate an active provision of commercial activity related to online safety technology (e.g. through the presence of a website / social media)
• provide Safety Tech products or services to the market (i.e. sell or enable the selling of these solutions to other customers)
• have identifiable revenue or employment within the UK

Section 2 of this report sets out the type of organisations within scope and sets out the products and services typically offered.

1.4 Methodology

We set out a simple overview of our methodology below. The full research methodology is included in appendices.

Stage	Description
Desk Research	Using a Grounded Theory approach, the team reviewed over 80 pieces of academic literature, sector overviews, and grey literature. We also long-listed all potential Safety Tech businesses within the UK and internationally, to identify the characteristics and offer of Safety Tech organisations.
Definition and Market Scoping	The research team developed a working definition for what constitutes ‘Online Safety Tech’. This also informed the development of a ‘Safety Tech’ taxonomy within this research, and the short-listing of Safety Tech firms i.e. those that provide a product or service aligned to the categories.
Workshops	We held four stakeholder workshops throughout this research including representatives from industry, academia, charities and policy-makers to test and agree the scope of this research.
One-to-One Consultations	This research is informed by 50 direct interview consultations with a range of Safety Tech stakeholders (in the UK and internationally), covering the views of industry, academia, charities and policy-makers. This included gathering feedback about the respective strengths, challenges and opportunities for the sector.
Market Analysis	We identified company trading information for the 70 Safety Tech providers using Bureau van Dijk FAME (for financial metrics), Beauhurst (a research platform that identifies high-growth and high-potential firms in the UK), and through direct consultation with industry consultees.

Given the emergence of the Safety Tech sector, data is limited in a number of areas, and will develop as the sector grows. Therefore, all references to revenue, market size or employment within this report are estimates only, using bottom-up modelling and direct stakeholder research.

2. Scoping Safety Tech

2.1 Introduction

Given the nature of this research, it is essential to set out the scope of what can be included within the ‘Safety Tech sector’, to provide an understanding of the products and services that are used to help make users safer online. This will help to distinguish the Safety Tech sector from parallel sectors - for example, cyber security is arguably more focused upon securing data and systems as opposed to people.

To define the Safety Tech sector, we need to:

• Understand the technical response utilised to reduce harm, such as detection and removal of illegal and harmful content, age-appropriate design or age-based safeguarding, detection of disinformation, and or content filtering etc

• Consider the type of harm involved that solutions seek to address, such as illegal video and image based content, hate speech, child exploitation, sexual material, personal harm, violence, bullying and harassment etc ^{[footnote 5]}

• Consider the type and extent of the risk involved: for example, identifying solutions that can detect and notify platforms and law enforcement about grooming behaviours online

• Identify those at most risk of harm, such as children and young people, or indeed - those who may not be aware they are exposed to harm e.g. disinformation or deep fakes etc; and

• Map the technologies and approaches deployed to counter the harm, for example, understanding the technical approaches deployed, such as risk detection and response through artificial intelligence (AI) or machine learning (ML) approaches

2.2 What is Safety Tech?

For the purposes of this research, we have adopted the following broad definition of the Safety Tech sector: Any organisation involved in developing technology or solutions to facilitate safer online experiences, and to protect users from harmful content, contact or conduct.

Within this research, we have noted that there is no unified or agreed definition of online Safety Technologies or Safety Tech, as described for the purposes of this report. However, when considering ‘online safety’ in the broadest sense, the risks can include:

• Content: being exposed to illegal, inappropriate or harmful material

• Contact: being subjected to harmful online interaction with other users; and

• Conduct: personal online behaviour that increases the likelihood of, or causes, harm

Therefore, when considering the Safety Tech sector, this research seeks to identify organisations that provide or implement technical products or solutions that either help to:

• Protect users from social harms when using technology and online platforms or services (typically through filtering or controls, or through detection and removal of potentially harmful content) or

• Provide mechanisms to flag, moderate, or intervene in the event of harmful incidents when using online platforms or services

“Ultimately, we’re about protecting children and vulnerable users. Safety Tech is working with those bounds – to focus on protecting people, rather than systems, from harm.” Safety Tech Start-up Founder

“Safety Tech as a term really resonates with what we do. When a new sector emerges, defining it can be an art. When you do something really new, you can look odd. When we go to cyber security events, we don’t look like a typical cyber security company. There’s something new and exciting here!” UK Safety Tech SME

For avoidance of doubt, this research seeks to identify and understand organisations that:

• Often work closely with law enforcement, to help trace, locate and facilitate the removal of illegal content online

• Work with social media, gaming, and content providers to identify harmful behaviour within their platforms

• Monitor, detect and share online harm threats with industry and law enforcement in real-time

• Develop trusted online platforms that are age-appropriate and provide parental reassurance for when children are online

• Verify and assure the age of users

• Actively identify and respond to instances of online harm, bullying, harassment and abuse

• Filter, block and flag harmful content at a network or device level

• Detect and disrupt false, misleading or harmful narratives; and

• Advise and support a community of moderators to identify and remove harmful content

2.3 Developing a Sector Taxonomy

Safety Tech is a fast-moving environment, with respect to the harms, risks and technologies in scope. Every day, the organisations involved are developing novel approaches to identify and counter risks before users are harmed. Therefore, this taxonomy is intended to reflect an overview of the current products and services offered by Safety Tech to inform a baseline analysis (2020) of the respective size, scale and potential of the sector, and to identify sector strengths and opportunities.

This taxonomy has been created and tested alongside DCMS and industry. This is not considered exhaustive and should be revisited on an ongoing basis to reflect the pace with which this sector is developing new solutions to existing and evolving harms. Please note that the order of the sub-categories does not imply any ranking regarding importance, size, or scale.

Safety Tech Taxonomy at a Glance

Level	Activity	How technology helps
01 At System Level	Automated identification and removal of illegal imagery	Identification and removal of known illegal child sexual exploitation and abuse (CSEA) and terrorist content (especially imagery and video), frequently through use of hashlists.
02 At Platform Level	Supporting human moderators	Detection of harmful or illegal content or behaviour (eg grooming, hate crime, harassment, suicide ideation, cyberbullying, extremism or advocacy of self-harm) and flagging to human moderators for action. Also, reduction of moderators’ own exposure to harmful content
—>	Enabling age-appropriate online experiences	Use of age-assurance and age-verification services to limit childrens’ exposure to harmful content.
03 At Device or Endpoint Level	User-initiated protection	User, parental or device-based products that can be installed on devices to help protect the user from harm)
—>	Network filtering	Products or services that actively filter content, through black-listing or blocking content perceived to be harmful)
04 In The Information Environment	Identifying and mitigating disinformation	Flagging of content with false, misleading and/or harmful narratives, through the provision of fact-checking and disruption of disinformation (e.g. flagging trusted sources).

Full Safety Tech Sector Taxonomy

HEADING	DESCRIPTION	HARM	APPROACH	BENEFIT	TECHNOLOGIES AND SERVICES	EXAMPLE ORGANISATIONS
1. At system level	Tracing, locating & removing illegal content	Terrorist content, Extreme/Revenge pornography, Child sexual abuse and exploitation	Detection and action against illegal content at system level	Tackling online crime, protecting citizens, preventing abuse	Hashing, URL lists, Takedown and domain alerts, Keyword collation and monitoring	IWF (iwf.org.uk), Qumodo (qumo.do), Cyan Forensics (cyanforensics.com)
2. At platform level	Platform level response to illegal content, including preventing illegal content from being published	Terrorist content, Extreme /Revenge pornography, Child sexual abuse and exploitation, Sharing of indecent images, Encouraging or assisting suicide, Harassment, Hate crime	Pre-moderation, detection, flagging and removal of illegal content at platform level	Protection from illegal online content	Threat detection and reporting, Platform monitoring, Hashing, Content filtering, Automated and human moderation, Image Processing, Computer Vision, Machine Learning	Dragonflai (dragonfl.ai)
—>	Platform moderation & monitoring: Prevention, detection & action against harmful conduct and / or content	Extremist content, Cyberbullying, Coercive behaviour, Intimidation, Violent content, Toxic Content, Advocacy of Self-Harm	Moderation and monitoring of harmful conduct and / or content	Protection from harmful online conduct and content	Threat detection and reporting, Platform monitoring, Hashing, Content filtering, Automated and human moderation, Image Processing, Computer Vision, Machine Learning	CRISP (crispthinking.com), Spirit AI (spiritai.com)
—>	Age oriented online safety - Age appropriate design	Age inappropriate content, Unsafe spaces	Safety by design	Design and development of user-centred online environments to keep children safe	Age appropriate web services, consent management	SuperAwesome (superawesome.com)
—>	Age oriented online safety - Age assurance	Age inappropriate content, Unsafe spaces	Age detection and verification	Protection from age-inappropriate content	Age assurance mechanisms: age estimation, e-IDs, database matching / attribute exchange	Yoti (yoti.com), TrustElevate (trustelevate.com)
3. At Device or End-point level	User initiated protection (user, parental or device-based)	Age inappropriate content, Unsafe spaces	Safety by design, age-based safeguarding	Creating safe online experiences for children	Endpoint protection software and applications	SafetoNet (safetonet.com)
—>	Network filtering	Extremist content, Cyberbullying, Coercive behaviour, Intimidation, Violent content, Harmful instruction	Detection and blocking access to harmful or inappropriate content	Preventing access to harmful material within defined settings	Content filtering and monitoring	smoothwall (smoothwall.com), Haandle Ltd (haandle.com)
4. In The Information Environment	Information governance: Detecting and disrupting false, misleading and/or harmful narratives	Misinformation, Disinformation	Fact checking, disinformation research and disruption	Ensuring citizen information accuracy and trust in the information environment and wider society	Disinformation research, Site assurance, AI/ML enabled automated fact-checking	Full Fact (fullfact.org), Factmata (factmata.com), Logically (logically.co.uk)
5. Other	Online safety professional services: Compliance & professional services	All	Compliance services, research, frameworks and methodologies for auditing, evaluating or mitigating potential harms	Enabling the development of safer online communities and embedding safety-by-default	Advisory support with implementing technical solutions	Island23 (island23.co.uk)

3. UK Safety Tech: Market Profile

Key Findings:

• There are at least 70 dedicated Safety Tech commercial providers based in the UK

• The UK market is home to a wide-reaching pool of expertise and capability, and covers the breadth of the Safety Tech taxonomy

• There are emerging Safety Tech clusters across the UK, particularly in London, Leeds, Edinburgh and Cambridge

This chapter sets out a summary of the market profile of the UK’s Safety Tech sector.

For research purposes, we have identified only:

• Organisations considered in scope with respect to the taxonomy

• Organisations that are active, and have a registered office located within the UK

• Organisations dedicated to providing Safety Tech products and services; and

• Organisations generating revenue or employment in relation to the provision of Safety Tech products and services

We also recognise that several of the larger tech companies are involved in the production or development of Safety Tech solutions (e.g. Microsoft’s PhotoDNA, AWS’ Rekognition Image Moderation API, Facebook and Google providing access to OpenAI and TensorFlow etc.). However, these providers are not measured within this sectoral analysis, which is focused on dedicated third-party Safety Tech providers.

The UK has a particularly active community of charitable and representative organisations involved in tackling issues relating to online safety. These are not included within the metrics in this analysis; however, key organisations are highlighted in this report.

This report is the first of its kind regarding this emerging sector, and therefore may not capture everything. That said, the following section highlights the contribution made by many of the UK’s leaders who help to make the online environment a safer place for all.

3.1 Scope

Using the Safety Tech definition and taxonomy, we have short-listed 70 organisations dedicated to providing relevant Safety Tech products and services which are registered within the UK.

There are also an additional 42 organisations that we have identified that are diversified (i.e. Safety Tech is part of what they provide) or are non-commercial in scope. In other words, the UK is home to over 100 organisations that specialise in supporting and developing online Safety Tech solutions.

For purposes of market analysis, we focus upon the 70 dedicated providers within the remainder of this report. However, we recognise that one of the UK’s strengths is that there are likely several providers within the marketplace that have the relevant skills and expertise to become more involved with Safety Tech in future years. These could include companies currently using AI, or designing cyber security solutions focused on threat intelligence that could arguably be deployed within the context of keeping users safer online for social purpose.

3.2 Products and Services

For the 70 organisations identified for commercial analysis, we have identified company descriptions of what they offer by means of web scraping and direct consultation.

The nature of this sector means that some organisations provide a wide range of products and services - for example, content moderation, sentiment analysis, and advisory services.

However, we have identified the best fit of each of the commercial organisations against the taxonomy categories, to illustrate the overall sector composition.

Taxonomy classification: Percentage of companies in each category

User protection	19%
Platform level	16%
System-wide governance	14%
Online Safety professional services	14%
Network filtering	14%
Age oriented online safety	14%
Information governance	9%

Typically, within a tech sectoral analysis – we might expect to see a majority of firms clustered in one sub-sector. However, the analysis suggests that the UK has, albeit from a small base of firms, a wide range of niche product and service capabilities available within the marketplace.

This is particularly important as it means that the UK market is home to a wide-reaching pool of expertise and capability, from which solutions can be deployed against a range of online harms for many different customer groups. For example, there are clear providers within the UK that offer technical solutions including:

• Detection and removal of illegal or harmful content e.g. through digital forensics provided to law enforcement

• User protection at the device level (e.g. ensuring that a phone or tablet is safer for a child to use) that can be pre-installed onto a device, or installed directly by a child, parent or guardian

• Provision of content moderation and threat intelligence expertise on a Business-to-Business (B2B) / Software-as-a-Service (SaaS) basis

• Network and content filtering sold to schools and the education sector; or

• Detection of disinformation and misleading narratives for consumers and media providers

Crisp - Protecting users, communities, brands and social platforms from online harmful content

Crisp stops toxic, harmful and fake online content from damaging enterprises, social platforms, democracy and public health. This content takes many forms, including fake news, terror propaganda, child-grooming, hate speech, medical disinformation, false rumours, threats, misinformation and many more online harms. Fueled by the increased popularity of messaging apps and closed social media groups, harmful content can now spread undetected until it reaches mainstream media channels.

Unfortunately, anyone has the power to create and share harmful content, especially bad actors (those with malicious intent) who distribute millions of new types every day. Crisp calls this the weaponisation of communication. Established in 2005 by online gaming and social media entrepreneur Adam Hildreth, Crisp began protecting children and teenagers using online games and social networks from abuse, sexual exploitation, cyberbullying and other online harms.

This relentless focus on stopping communication from being weaponised has been Crisp’s mission from day one. Today that passion extends to defending enterprises, brands, publishers and social platforms around the world.

Trained on 15+ years of online harm detection, Crisp’s real-time knowledge graph discovers, tracks, rapidly alerts and removes harmful content that is created by bad actors and shared across the open, deep, dark social web and messaging apps. This technology is used by a global team of experts specialising in signals intelligence, linguistics, big data, AI, human intelligence, psychology, data protection, and law and regulation. It has become an early warning system for stopping harmful content 24/7/365. As a result, Crisp contributes to the safe, daily online experiences for over two billion users (covering an estimated 400 million children) across the globe.

Since its establishment, Crisp has secured over £30m in external investment, and continues to rapidly expand while proudly keeping its roots firmly in Yorkshire.

3.3 Location

This section sets out the registered location of the identified Safety Tech organisations.

London (39%) and the South East (13%) reflect most firms in scope. It is also worth noting that there is significant presence within Yorkshire and the Humber (10%) in relation to its size. This is reflected within the breakdown by UK region (below) and a UK heatmap (below).

When exploring the economic contribution of the Safety Tech sector, we note the emergence of Safety Tech clusters particularly in London, Leeds, Edinburgh and Cambridge; however, Safety Tech talent is relatively well dispersed across the UK, and is clearly supporting the growth of tech sectors across the nation.

Percentage of registered UK organisations by region

London	39%
South East	13%
Yorkshire & the Humber	10%
Southwest	7%
Northwest	7%
Scotland	6%
East of England	6%
Wales	4%
West Midlands	3%
East Midlands	3%
Northern Ireland	1%
North East	1%

UK Safety Tech hotspots

Smoothwall - Protecting children in the UK and globally

Smoothwall, based in Leeds, has been operating for almost 20 years and provides filtering and safeguarding solutions to the education and public sector. Over one in three schools in the UK use Smoothwall solutions to keep their students safe from online harm. Last year, it generated almost £15m in revenue.

Smoothwall’s solutions include tools that:

• Filter harmful content, and detect online content that is inappropriate for children. This provides schools with real-time content analysis, device and social media controls;
• Support safeguarding and education, helping schools to meet requirements for appropriate monitoring and record-keeping; and
• Support with firewalls, and e-Safety training resources.

Smoothwall has been able to draw upon its experience and expertise to support education providers globally, and currently works with over 30,000 IT leaders to protect the safety of more than four million students online. It works across Europe, the United States, South America, Africa, Asia and Australia.

4. Economic Contribution of the Safety Tech Sector

4.1 Introduction

This section outlines how the Safety Tech sector has grown in recent years, including an estimation of sectoral revenue and employment.

We note that many of the organisations identified are at a ‘pre-revenue stage’ and are investing in product development, or are micro firms (which do not have to provide full audited accounts annually). However, we have used estimated total sectoral revenues using company accounts, direct consultations, and company-level estimation as appropriate.

Key Findings:

• The sector is relatively young, with the number of dedicated online safety firms almost doubling in the last five years

• Total sectoral revenue (for the last financial year) was an estimated £226m

• Total employment (for the dedicated providers) is an estimated 1,700 FTEs in the UK

• Revenues have grown year-on-year by 35% per annum since 2016. Although it is important to recognise this does reflect a small number of firms, this growth rate is particularly strong when compared to other emerging sectors.

• There are powerful drivers for growth within the Safety Tech sector, and our modelling suggests that the sector could reach £1bn in annual revenues by the mid-2020s

4.2 Company Registrations

The figure below sets out the incorporation date (when the business started) of the 70 dedicated Safety Tech providers, to provide an overview of the relative age of these businesses. This highlights the significant increase in the number of dedicated online safety organisations, which has almost doubled in the last five years.

Despite the overarching Safety Tech sector being in its infancy, there are examples of early-stage companies that have demonstrated high-growth relatively quickly following establishment e.g. SuperAwesome, Crisp and Yoti.

Date of Incorporation (running sum)

UK Safety Tech Timeline (1997 - 2020)

Year	Event
1997	Internet Watch Foundation established
2001	Smoothwall established to provide web filtering for schools
2002	Emergence of content moderation in the UK: Tempero, The Social Element founded
2005	Technology for detecting visual threats and image content - Image Analyser founded in the UK; Crisp founded in Leeds - Stopping harmful content from damaging enterprises, social platforms, democracy and public health.
2008	UK Council for Child Internet Safety founded
2009	Full Fact founded in the UK to enhance the quality of public debate and access to fact-checked information; eSafe founded
2011	CameraForensics founded - their platform is now used by law enforcement agencies globally to identify and safeguard victims of abuse
2013	Emergence of some of the UK’s largest Safety Tech firms: SuperAwesome, SafeToNet, Gooseberry Planet
2014	Yoti founded; Internet Matters launches in May 2014; The UK Prime Minister hosts the first #WePROTECT Global Summit in December 2014
2015	Spirit AI founded - using AI to combat toxic behaviour in gaming platforms and communities
2016	Cyan Forensics, Qumodo, Vigil AI, AgeChecked, Securium founded
2017	Logically, Factmata founded
2018	GDPR came into force in May 2018; Unitary, Astroscreen, DragonflAI founded; Trust Elevate graduates from the NCSC Cyber Accelerator; UK Council for Internet Safety launches
2019	New entrants to market e.g. Cap Certified; Online Harms White Paper published; Record Year for Safety Tech Investment (£50.6m)
2020	Government publishes interim response to Online Harms White Paper; Online Safety Tech Industry Association (OSTIA) founded

4.3 Estimated Company Size

Of the 70 organisations identified for commercial analysis, the majority are micro or small firms (59% and 31% respectively).

Category	Definition	Number of Firms	%
Large Company	Employees >=250; and Turnover > €50m or Balance sheet total > €43m	1	1%
Medium Company	Employees >50 and < 250; and Turnover <= €50m or Balance sheet total <= €43m	6	9%
Small Company	Employees >10 and < 50; and <= €10m or Balance sheet total <= €43m	22	31%
Micro Company	Employees < 10; And Turnover <= €2m or Balance sheet total <= €2m	41	59%
Total	All	70	100%

4.4 Estimated Employment

We estimate that there are approximately 1,700 people working within dedicated Safety Tech firms based in the UK (dedicated). This has been gathered through company accounts and consultation data. We understand that this figure is likely to be higher should charitable organisations, internal moderation teams, and larger tech firms be included in future scoping of the sector.

Some of the largest Safety Tech employers in the UK include Crisp (over 300), Yoti (over 220 in the UK), SuperAwesome (>150), and Smoothwall (>125).

We estimate there are 1,700 FTEs working in dedicated UK Safety Tech providers.

Estimated Safety Tech Employment (Dedicated) by Region

London	743 (44%)
Yorkshire & the Humber	577 (34%)
East Midlands	87 (5%)
South East	63 (4%)
East of England	60 (4%)
Northwest	44 (3%)
Wales	36 (2%)
Scotland	26 (2%)
West Midlands	21 (1%)
Northern Ireland	18 (1%)
Southwest	15 (1%)
North East	9 (1%)

[Chart shows percentage of UK employed (registered level) - Source Perspective Economics, BvD FAME (n=70)]

4.5 Estimated Revenue for the Sector

As online safety has become a greater policy focus in recent years, the market has also shown considerable growth and activity.

We estimate that in total, the Safety Tech sector generated £226m in annual revenues in 2019, and it has grown rapidly with an estimated 35% annual growth rate ^{[footnote 6]} since 2016.

A number of companies have seen even higher rates of growth. Our review of company accounts suggests that some of the most established Safety Tech companies in the UK (those earning in excess of £5m) have in recent years grown at rates of up to 90% per annum, and reached profitability. It is anticipated that domestic and international demand will continue to grow rapidly in the years ahead.

Whilst it is important to recognise this does reflect a small number of firms, this growth rate is particularly strong when compared to other emerging sectors. For example, Tech Nation estimate the digital sector in the UK to have grown by 4.5% in 2018 ^{[footnote 7]} and DCMS’ UK Cyber Security Sectoral Analysis (2020) suggests the UK cyber security sector has grown at a rate of c.20% per annum since 2016.

SuperAwesome: Global leaders in Kidtech

“In 2013, we started as a handful of people in a room, mostly being turned down by investors,” SuperAwesome cofounder and CEO Dylan Collins said. “Today our kid tech platform enables over 12 billion kid-safe transactions every month, ranging from advertising and video to community and parental consent.” VentureBeat, 2020

SuperAwesome’s kidtech platform is used by hundreds of companies to ensure that digital engagement with children is safe, private and compliant with laws such as COPPA, GDPR-K, CCPA and others. Today, SuperAwesome powers over 12 billion kid-safe digital engagements every month across safe advertising, video, community and parental consent.

Used by over 300 of the world’s top kids brands and content owners, its technology helps to:

• Intercept and strip out personal data that are routinely collected from children viewing ad-supported content;
• Enable responsible publishers to age gate their content, and also automates the process of obtaining and managing parental consent; and
• Provide a kid-safe alternative to mainstream social media platforms, with AI moderation and human moderation helping to ensure content is safe and appropriate for kids.

Founded by serial entrepreneur Dylan Collins, the company is backed by investors including Microsoft’s M12 Ventures, Mayfair Equity Partners, Hoxton Ventures and Harbert European Growth. SuperAwesome has offices in London, New York, LA and Chicago. It also holds investments in TotallyAwesome (Southeast Asia) and Kids Corp (Latam).

4.6 Drivers for Growth in the UK Safety Tech Sector

Throughout the consultation process with sector stakeholders, the research team sought to explore the growth rate of 35% per annum, and endeavoured to gather views among established and early-stage companies as to whether they thought this reflected their experience of the market to date. Additionally, the intention was to gather opinion regarding the revenue potential of the sector in future years.

There is emerging evidence that as some of the Safety Tech firms have reached a certain level of maturity, they have been able to secure higher growth rates and rapidly expand within the market. Overall, respondents noted that the growth to date has been achieved despite the relative nascence of the sector, and expressed confidence that the growth rate to date is likely to continue, or exceed this 35% per annum rate.

There are a wide number of driving factors for this rapid growth, and several of these invoke the real need to deploy technical solutions to address onset societal harms. These include:

Increased consumer awareness regarding online harms, alongside an expressed loss in trust of many consumers within tech platforms

Throughout the consultations, it was noted that customers have become much more knowledgeable regarding their online rights particularly through the introduction of GDPR. This, combined with concerns with privacy and loss of trust e.g. following the Cambridge Analytica data scandal, has meant that online safety is much higher on consumer agendas.

Market preference to minimise toxicity and abuse from platforms, and to disassociate platforms from harmful or damaging material

There has been a recognition from platforms within the market that abuse and toxicity on platforms actually has a material impact upon how consumers engage with platforms. For example, if an online game is full of abuse and is poorly moderated, that can result in a material loss in players and revenue. Therefore, there has been a more concerted effort among platforms to engage with content moderator firms to help address these challenges.

Enhanced legislative and regulatory considerations, particularly in tackling illegal and harmful content – and the need for policy to reflect new forms of online harms such as livestreaming

Globally, there has been a recognition that regulation must be introduced to help tackle long-standing issues of online harms. Within the UK, the Online Harms White Paper is likely to act as a catalyst for enhanced demand for Safety Tech products and services – indeed, the UK is not alone in this push, as reflected internationally through examples such as Australia’s establishment of the eSafety Commissioner (2015), and Germany’s Network Enforcement Act (NetzDG) in 2017 that requires online platforms to remove ‘manifestly unlawful’ content or risk fines.

Personal exposure to harms, and the significant impacts these can have – and a desire to actively challenge and counter this harm

Within this research, there are a number of businesses that were established as a direct result of parents finding their own children exposed to online harm, and not wanting this to happen to others. There is considerable passion and commitment within this sector to protect users online, and to make the online world a better place.

The increasing volume of content online requires a move towards using AI in the content moderation process

There is a consensus that online content is being created faster and in such volumes than can currently be solely screened or moderated by humans. Use of new solutions such as AI and Natural Language Processing can increasingly help human moderators to better detect harmful content, material, or sentiment in real-time, and to also reduce moderators’ own exposure to harmful content.

Heightened concerns regarding the spread of disinformation in the public domain

Disinformation is a commonly used tool online, often deployed by sophisticated threat actors to generate false narratives. Countering misinformation and disinformation and providing transparency in the reporting of information is critical to a democratic society.

Collectively, these are powerful drivers that industry expects will sustain demand to come for Safety Tech products and services.

4.7 Growth Scenarios

Given the growth drivers, the sector is considered likely to see double-digit growth in the years ahead.

Whilst we recognise that it may be too early to project future revenues, based on assumptive growth rates suggest the sector may reach over £1bn in annual revenues by the mid-2020s.

This is illustrated in the following graph, which models 20% (low-growth scenario), 35% (current growth trajectory) and 50% (increased from 2021, reflected growing drivers) rates.

We note the considerable economic impact of COVID-19 on the global economy, there will also be enhanced demand opportunities for Safety Tech provision. For example, we are likely to see increased demand for AI-driven techniques to support human moderators.

Estimated revenues by 2024

5. Investment and Funding Landscape

5.1 Introduction

This section sets out an overview of the investment landscape for the firms identified within this sectoral analysis.

To inform this analysis, all the organisations identified were input into the Beauhurst platform. Beauhurst (www.beauhurst.com) tracks announced and private investments, and the performance of high-growth companies in the UK. It also monitors UK business participation within well-known business accelerator and incubator initiatives, and tracks where businesses have secured funding from public bodies such as Innovate UK.

Of the dedicated organisations identified within this sectoral analysis (n=70), 29 (41%) were flagged or tracked within the Beauhurst database.

Given that there are only approximately 50,000 businesses that meet the Beauhurst tracking criteria in the UK, this signals that the Safety Tech sector consists of a significant proportion of high-growth and high-potential firms, reflected by investment activity to date.

Key Findings:

• The Safety Tech sector has raised over £136m in external investment in the last five years

• 2019 was a record year for Safety Tech investment, with £51m raised across 19 deals

• In 2019, the combined value of UK Safety Tech companies (that have received known investment) was £503m

• It is considered likely that a UK Safety Tech unicorn (a privately held company with a valuation of over $1bn) could emerge in the near future

5.2 Investment Activity to Date

Within the last five years, the volume and value of external investment within Safety Tech companies has grown significantly.

In 2015, the Safety Tech sector raised £6m in external investment across ten deals. By 2019, this has increased more than eight-fold to £51m across 19 deals, as the scale and maturity of companies within the sector has developed.

In addition, Safety Tech companies raised £16m in Q1 2020 (Jan-Mar 2020) (not shown in the graph below).

Overall, tracking all investments over the last five years demonstrates the surge in investment secured by the Safety Tech sector, particularly since 2017.

Within these investments, there are a notable number of high-growth success stories within the sector such as SuperAwesome (£46m over six fundraisings), SafeToNet (£23m, seven fundraisings), Crisp (£21m, five fundraisings), and Yoti (£19.4m, four fundraisings).

Whilst the total number of deals fell in 2019, this is a recognised trend which is consistent across all (or most) sectors and is usually explained as being due to a shift up the value chain as investors prefer to invest later in the life of a company.

Summary of investment to date

5.3 Investment Analysis

The following subsections set out an overview of Safety Tech investment by stage of evolution (maturity), taxonomy classification, and location.

Seed

A young company with a small team, low valuation and funding received (low for its sector), uncertain product-market fit or just getting started with the process of getting regulatory approval. Funding likely to come from grant-awarding bodies, equity crowdfunding and business angels.

Venture

A company that has been around for a few years, has either got significant traction, technology or regulatory approval progression and funding received and valuation both in the millions. Funding likely to come from venture capital firms.

Growth

Company that has been around for 5+ years, has multiple offices or branches (often across the world), has either got substantial revenues, some profit, highly valuable technology or secured regulatory approval, significant traction, technology or regulatory approval progression, funding received and valuation both in the millions. Funding likely to come from venture capital firms, corporates, asset management firms, mezzanine lenders.

Source: Beauhurst definition

Investment by Stage of Evolution

Since 2015, the majority of investment raised by Safety Tech companies has been raised by companies at growth stage (64%, £98.4m).

However, segmenting all deals by the stage of evolution at the time of the deal (n = 83), demonstrates that there is relatively active seed level investment within the Safety Tech market, which subsequently (as demonstrated by several Safety Tech firms in recent years) has been able to secure venture and growth level investments.

Volume of deals

Stage of Evolution	Number of deals (%age of total)
Seed	34 (41%)
Growth	14 (17%)
Venture	35 (42%)

Value of deals

Stage of Evolution	Value of deals (%age of total)
Seed	£11.1m (28%
Growth	£98.4m (64%)
Venture	£43.2m (28%)

Taxonomy Category

Exploring investments by each of the Safety Tech taxonomy categories (by value) demonstrates that investment activity spans all of these categories. Age Orientated Online Safety (e.g. Age Assurance and Age-Appropriate by Design provision) is the leading category for investment, reflecting the recent successes of firms such as SuperAwesome and Yoti in securing external investment.

All of the remaining taxonomy categories have all demonstrated emerging maturity with respect to the volume and value of external investment.

Safety tech investment, by taxonomy category

“We’ve seen a real change in demand since 2016 – particularly among the gaming industry, that has moved away from ‘why would you want to filter’ to realising that social interaction matters to their platforms.”– UK content moderation provider

Investment by Region

The following diagram sets out an overview of investment (value) within dedicated Safety Tech providers (since 2015) by region. This demonstrates that London was the most active region for investment (covering 57% of the number of fundraisings and reflecting 73% of total investment made). Yorkshire and the Humber also performed strongly with 17% of total investment made.

The remaining investment was distributed across Scotland, Wales, North West, South East and the East of England; however, whilst these regions reflect a significant proportion of the number of fundraisings (36% combined), they involve a much lower proportion of the value raised (10%).

Leading regions for Safety Tech investment

5.4 Company Valuations

In total, the combined post-money valuation value (2019) of UK Safety Tech companies is £503m.

Although the Safety Tech sector is still early-stage regarding income and revenue generation, our findings demonstrate that it is a sector in which firms and investors see high growth value in years ahead.

This means that the potential economic value of Safety Tech is unlikely to be captured by measures such as current revenue, Gross Value Added, or employment.

On this basis, we have also sought to identify the most recent post-money valuation value of companies that have secured external investment. This is typically based upon the stake taken in a company. For example, if a company sells 10% of its equity for £5m, its assumed firm-level valuation will be £50m.

Using Beauhurst investment data, there are 24 companies identified (within the 70 commercial providers ^{[footnote 8]} ) with a known post-money valuation. Where companies have not secured external investment, or have sought not to disclose investments, it is not possible at this stage to determine their market value.

This is likely to be a conservative figure. Consultation with a number of the largest Safety Tech firms, suggested that several of these firm-level valuations are likely to be considerably higher as of March 2020.

There is a consensus that the UK is likely to see its first Safety Tech unicorn (i.e. a privately held company worth over $1bn US dollars) emerge in the near future, with a small number of other firms also demonstrating the potential to hit unicorn status by the mid-2020s.

Unicorns reflect their name – they are rare, and the UK has to date created 77 unicorn businesses (as of Q4 2019). Therefore, the potential to create one or more unicorns in the years to come demonstrates the significant high potential of the Safety Tech market as understood by the investment community.

5.5 Funders

Of the investment identified (£84.8m), many of the funding sources are undisclosed or unavailable. However, consultation with a number of the businesses in receipt of undisclosed investments suggests that a significant amount of investment is generated by high-worth private investors using the Enterprise Investment Scheme (EIS).

The remaining investment activity comprises largely of:

• Private equity and venture capital (£38.3m) raised

• Business Angels and private investors (£25.3m raised)

We anticipate that private equity and venture capital will expand as the sector grows, and more companies seek Series (A – C) investment rounds.

Consultees also noted that there is a growing amount of ESG (Environmental, Social and Governance) investment aligned to Safety Tech investment in the UK. This includes investors seeking to make investments that have a clear social impact such as protecting children from online harm. Indeed, it is estimated that almost half of UK investors ^{[footnote 9]} expect to increase their ESG investments over the next few years – demonstrating the potential of Safety Tech investments as generating commercial outcomes, but achieving social good.

The funding data also demonstrates that Crowd Funding, Devolved and Local Government, Angel Networks, and Banks all have a role to play in early stage investment; albeit the sums raised to date have been relatively small.

Breakdown of Funders

Funder type	Seed	Venture	Growth
Undisclosed investors	£6.5m	£20.7m	£35.6m
Private equity and venture capital	£1.5m	£6.1m	£30.7m
Business angels	£2m	£10.3m	£13m
Unknown	0	£0.8m	£19m
Angel network	0	£0.8m	0
Crowd funding	0	£0.8m	0
Bank	0	£0.2m	0
Local and regional government	£0.2m	0	0
Charity/not-for-profit company	£0.1m	0	0

[Source: Beauhurst (n=84, n=£124.8m)]

5.6 Grants and Support

Beauhurst also identifies where companies have been in receipt of grants from public bodies. We have identified 15 grants (since 2015) received by ten dedicated online safety companies, which total just under £1.8m.

When examined against the taxonomy, grants are much more likely to be awarded to companies within the ‘system-wide governance’ and ‘platform governance’ space, which reflects recent Government initiatives to support the development of innovative technologies.

This includes, for example, the GovTech Catalyst scheme (delivered in 2018) to develop technology that can automate the detection of imagery produced by Daesh and other terrorist groups.

It is significant to note that whilst these ten firms have received £1.8m in grants, they have subsequently raised a further £29.3m in external investment in recent years. This reflects the importance of Government continuing to provide financial grants and challenge funding, as small grants can often help to facilitate network development, pitching skills, and help smaller firms to secure external investment.

In consultation with the Safety Tech sector, key feedback regarding grants and support for the sector included:

A need for enhanced and dedicated Safety Tech grant initiatives

A common view expressed among the sector is that although knowledge of bodies such as Innovate UK is strong, there has to date been no formal ‘Safety Tech’ grant mechanism – and designing such an initiative would be helpful to demonstrate there will be dedicated support to grow this sector specifically.

Significant value in focused grants and the role of public procurement in supporting early-stage businesses make their first steps as a commercial venture

For several early-stage Safety Tech start-ups, the potential to engage with a grant or SBRI type initiative to develop a solution or prototype (to what is often a public policy issue) is inherently valuable, as it demonstrates market potential and provides an opportunity to test products in an applied and real-world way.

Grants and investments can provide external validation that a product or service works or has high potential

As one respondent, a UK Safety Tech SME exporting to EU markets, noted:

Being able to take part in a government backed grant or programme, shows that we have been able to work with Government. This is incredibly valuable as we seek to export or work with others (and establish contracts)

Grants can also provide the key stimulus to bring the right mix of organisations into the same room, or jointly meet the same challenges.

5.7 Views on the Investment Landscape

During the Safety Tech one-to-one consultations, we asked businesses for their views on challenges or barriers in securing external investment. The main findings included:

Although investment activity in UK Safety Tech has increased rapidly in recent years, we are still early in the journey

Consultees noted that Safety Tech has become more well-known and recognised by investors, and that the emergence of new regulations around Online Harms has helped to catalyse the sector. However, there was a broad consensus that this is still a narrative that needs to be further defined, explained and pitched to investors to increase total investment.

Overcoming the challenge in positioning

Safety Tech companies within the UK are passionate about what they offer, and how their solutions can help protect users online. This means that when securing external investment - often to develop or bring a new product to market, they require an investment partner who shares their values, and who understands the need to mix profit and returns, with purpose and ethics.

This means that companies often need to be selective in the type of investor they engage with. For example, some companies choose only to raise investment from known high-worth individuals, or patient capital schemes.

There was also strong sentiment that Safety Tech can often be viewed as ‘soft’ or ‘less commercial’ than cyber security or RegTech by investors. Any initiatives to break down such barriers, and demonstrate the tangible benefit of investing in such businesses would be particularly welcome.

“We’ve had investors ask us ‘How much money will you make me in ten years?’. I might not be able to answer that, but I can tell you how many children we’ll have helped by then.”– UK SafetyTech Provider’s experience of the investment landscape (has since raised over £20m in external funding)

However, we note that there has been considerable growth in social impact and ESG investing in the UK and overseas, and this has been cited by several consultees as a significant contributing factor to their capital raising.

Greater support for early-stage companies would be welcomed: There was significant demand and interest for a Safety Tech-specific ‘accelerator’ initiative that could help early-stage Safety Tech companies receive advice, mentorship, and point them in the direction of suitable investors and patient capital.

“Let’s have a clear mission – find the best companies, and accelerate them.” – UK Safety Tech start-up (focusing on AI techniques)

“We’re a small start-up, with a unique proposition. We’re trying to self-fund, but we need support – we don’t want larger tech firms being able to squeeze us out of the market before our product is ready.” – UK Safety Tech start-up (positioning as RegTech)

“The UK Government needs a huge pat on the back regarding tax relief schemes (EIS). This makes a game changing significantly competitive advantage for the Safety Tech sector.” – Established Safety Tech businesses – funded through high worth individuals

Tax relief schemes play a significant role and should be maintained

Several Safety Tech companies (as reflected by the volume of undisclosed investment within the previous section) have been in receipt of funding from high worth individuals – who recognise the need to be patient within this industry, but also the commercial viability of the sector. Consultees noted that the Government-backed Enterprise Investment Scheme (EIS) for tax relief was of significant benefit to encourage such investment and should be maintained.

Incentivising adoption and demand

Consultees also noted that the best way to stimulate investment in the UK is to put in place incentives for adoption and implementation of Safety Tech solutions, for example, through tax credits or a regulatory approach to enable spending on adoption of services.

SafeToNet: Protecting children ‘in-the-moment’

SafeToNet’s technology educates and safeguards children “in-the-moment” as they use their device, such as a tablet or mobile phone. It is a ‘safeguarding assistant’ that includes a smart keyboard that detects and filters risk in real-time steering children away from trouble by filtering harmful outgoing messages before they can be sent and any damage can be done. SafeToNet helps to prevent sexting, bullying, abuse and aggression.

The software respects the child’s rights to privacy and ensures no-one can see what is being typed, sent or received. The smart keyboard provides children with immediate feedback as they type, recognising signs of low self-esteem, doubt and dark thoughts. When the software detects that a child may be in trouble or is anxious or fearful, it provides audio guides that provide reassurance and advice ‘in-the-moment’ and when the child needs it the most.

Commercially, SafeToNet has found particular success in the UK, Germany and the Middle East, and has raised significant external investment (over £23m) in recent years.

SafeToNet also recognises the value in supporting external projects and initiatives that focus on safeguarding children from risk and harmful content. Its founders established the SafeToNet Foundation, a charity that provides free software licences, grants and financial support to such initiatives, and improves the education and awareness of online safety issues such as screen-time and grooming among children, parents and schools.

6. Research, Innovation, and Collaboration

6.1 Introduction

The UK’s research strengths have had a direct role in supporting innovation along with developing and testing robust and trusted Safety Tech solutions.

Relationships between academia, government, charities, and international organisations must be supported to ensure that UK Safety Tech products and services continue to be as effective as possible, and reach and benefit a wide user base.

“There are some really innovative tech firms in the UK that may not view themselves as online safety, but we would want in this space.” - Stakeholder Feedback with UK SafetyTech workshop (January 2020)

6.2 Safety Tech Strengths

We asked stakeholders about their views on research, innovation and collaboration within the UK Safety Tech market.

The overriding theme is that the UK’s strengths across fields such as emerging technologies, social investment and innovative regulation have created an environment which is particularly conducive to the development of Safety Tech.

In particular, the UK was seen to have:

• A depth of experience in online safety: Organisations such as NSPCC, Royal Taskforce on Cyberbullying, IWF and Childline have a strong understanding of ‘what works’ in online safety provision, and the issue is gaining traction in the gaming industry

• A dynamic and disruptive tech sector: The growth of the online safety market in recent years has been underpinned by the UK’s broad base of young, innovative tech firms with disruptive ideas

• Commercial and technical strengths in aligned disciplines such as Cyber Security, AI, machine learning, RegTech and Governance; and

• World-leading academic and research strengths regarding legal, behavioural and criminological aspects of online harm – these are all vital with respect to understanding online harms, the forms they take, and how harms can be mitigated and reduced

6.3 Research Capabilities

The UK is home to a strong academic community with respect to Safety Tech, and some of the most cited universities included Cardiff University (HateLab), University of Sheffield, Oxford University (OII), University College London, Abertay University and the University of East London.

In consultation with the Safety Tech sector and academia, some of the key findings included:

UK has world-leading academic capacity regarding online harms. However, there is a need for interdisciplinarity – to bring together behavioural, technical, legal, and ethical strengths

Consultees ^{[footnote 10]} noted:

The UK has always continued to be a really important academic sector. There’s plenty of skills and knowledge. Anyone involved in cyber security or FinTech is likely to understand trust, identity and security – so having that technical knowledge within the UK is genuinely helpful.

There’s an argument for creating a narrative that online safety is a spin-off of our cyber security sector. We’ve got the technical fundamentals. Now, let’s drop the humanities into tech. In global terms, we very often take a leading role in child protection.

It is essential to put money into academia to grow. We need more young people in this space – they know about harms on the internet, at a much deeper level. If you start younger researchers in an academic context, they may then be motivated to start a company.

The UK’s approach to privacy and public safety gives it a good edge in research. Thinking of counter-terrorism, we’ve experienced what it is to see terrorist incidents, so we can have that academic discussion on privacy and safety.

There is a need for specific Online Safety or Online Harms research calls, which would act as a signal internationally regarding the size and scale of the UK’s Safety Tech community.

Several consultees noted the need for UKRI, Innovate UK and partners to work closely with the sector to design research and projects around online harms and safety, rather than include ‘online safety’ research calls alongside or within ‘cyber security’ themes.

Funding should be available to bring together industry and academia on collaborative projects

There is a real desire among Safety Tech providers for Government to show sufficient ambition and deploy resources for departments and the regulator accordingly to invest in projects. These need to span industry (reflecting match funding), academia, and charities – and to be as inclusive as possible.

6.4 Spin-outs

An academic spinout is a company that set up to exploit intellectual property developed by a recognised UK university, or where a university owns IP licenced to the company. The UK is home to some notable Safety Tech spin-outs, including Cyan Forensics (Edinburgh Napier University), Raven Science (City, University of London), and Securium (University of Surrey).

Some stakeholders noted that tailored support to academic start-ups (similar to the Cyber Security Academic Start-Ups Programme) would be particularly useful to help encourage innovation and further commercialisation within the Safety Tech sector.

Cyan Forensics: Leading the fight against online child sexual abuse imagery.

Cyan Forensics was founded in 2016, as a spin-out from Edinburgh Napier University. Its digital forensics technology can identify terrorist content or child sexual abuse material on devices within minutes, minimising forensic analysts’ time and allowing police to make decisions quickly and confidently. It can also help social media companies and cloud providers to find and block harmful content.

Quicker searches and identification of child sex offenders not only saves police time and money – it means that vulnerable children can be safeguarded faster, and suspects brought to justice sooner.

The company has recently signed a partnership with the National Centre for Missing and Exploited Children in the United States, and was a winner of the prestigious PitchGovTech competition at the GovTech Summit in Paris. It follows its recent contract with the UK’s Home Office, which will see the technology being used by the Child Abuse Image Database (CAID) and rolled out to police forces nationally.

Cyan Forensics is one of Scotland’s highest growth tech firms, and has secured over £3m in external investment.

“We’re excited to have raised the funds which will now allow Cyan Forensics to deliver the next stage in our significant growth plan…. a key focus will be expanding our reach within the UK as a result of our partnership with the Home Office and into international markets.” Ian Stevenson, CEO

Internet Watch Foundation: Leading the international fight against illegal content

“The IWF sits at the heart of the national response to combating the proliferation of indecent images of children. It is an organisation that deserves to be publicly acknowledged as being a vital part of how, and why, comparatively little child sexual abuse material is hosted in the UK.” - Independent Inquiry into Child Sexual Abuse (IICSA) (2020)

The Internet Watch Foundation (IWF) was founded in 1996, and since then has led the way in removing significant volume of child sexual abuse material hosted online. Its vision is for the global elimination of child sexual abuse imagery online.

It helps victims of child sexual abuse worldwide by identifying and removing online images and videos of their abuse. It also provides a hotline and portal for the public to report illegal content anonymously.

The IWF is a unique organisation in the fight against illegal content online, as it works closely with Government, industry, internet companies, social media, and law enforcement to actively trace, locate, and remove content. Through its experience of more than 20 years, it brings technical and moral expertise.

More than 140 companies are members of the IWF, from global big tech firms through to small filtering companies based in the UK. As a non-profit charity, the IWF has a membership model that provides funding to directly support its vital work of removing child sexual abuse content from the Internet. This also provides members with a range of tools to combat this content, including:

• Image Hash Lists (with Microsoft PhotoDNA) to help organisations stop the upload, sharing, or use of child sexual abuse images or videos on their platforms

• Takedown Notices and Simultaneous Alerts to receive immediate alerts about illegal content hosted on networks;

• URL & Domain Alerts to share webpages that may contain child sexual abuse material so that websites and domains can be blocked and removed

• Keyword Lists – a unique list of cryptic terms associated with child sexual abuse material – this helps monitor networks and applications, and improve search results

Through working alongside a range of organisations responsible for network infrastructure, including ISPs, domains, filterers, hosting, search engines, social media, gaming, and mobile operators – the IWF have helped to radically reduce the volume of child sexual abuse content hosted in the UK, and continue to lead the fight.

6.5 Collaboration & Access

It is clear that UK Safety Tech organisations seek to be collaborative and recognise that the sector requires input and support from industry, academic, government, charities, and the public to grow.

Within the consultations, we asked organisations what barriers existed in collaboration or accessing support for growth. This subsection summarises some of this feedback.
Need to improve awareness, education, and encourage deployment of ‘Safety by Design’ in consumer products and platforms:

Stakeholders noted a need to educate users and parents that there are solutions that can be deployed to tackle online harms, and these issues are not always unsolvable. Enhancing awareness of online safety solutions would be particularly welcome, but also doing so in an ‘instinctive and natural way’. For example, initiatives to make sure that technology such as mobile phones or tablets are ’safe-out-of-the box’ could be a useful mechanism for increasing the reach and implementation of Safety Tech solutions.

A need for safe and secure access routes to public datasets

Several businesses cited their need for safe and secure access to datasets illustrating online harms, which they could use to train, test and improve machine learning systems. The value of enabling greater access (with required security clearances) to the Home Office-led Child Abuse Image Database was particularly emphasised. This would support UK firms to test the quality of their image or video detection platforms through submitting algorithms and identifying the effectiveness of their approaches e.g. percentage of false positives identified in detecting material.

A collaborative approach to developing solutions, bringing together larger tech providers, start-ups and government, was also felt to be of benefit in growing confidence in stakeholders that technical solutions to issues such as livestreaming can be deployed on platforms.

“Opening up access to data would be amazing. We could move away from identifying and reporting crime – but towards opening this up to researchers and identifying solutions that work.” - UK Safety Tech start-up (identifying potential for UK datasets)

Provision and access to curated and synthetic datasets, as well as hackathon type initiatives were also cited as a key parts of supporting early-stage companies test their solutions and raise awareness.

Growing a community of practice

Several consultees noted the need for regular roundtables bringing industry together with Government to raise awareness, share ideas, pitch and enable discussion, and to promote access to schemes such as accelerators and support for the sector.

This should also seek to include not only suppliers of Safety Tech solutions, but those involved in implementing solutions at an industry level. In this respect, community engagement is critical to the long-term success of UK Safety Tech.

Consultees felt that Government backing for such roundtable initiatives would clearly signal that UK Safety Tech is a key market, and help to set out Government’s vision for the sector as a growth area for the UK economy, with a critical mass of validated, trusted and experienced providers.

Driving international collaboration

Several consultees also noted that international research and collaboration should be a priority for the UK, including research centres dedicated to this space. For example, the UK is home to the Global Disinformation Index, which receives UK Government support to identify misleading and disinformation narratives.

Ensuring greater international engagement through shared research, use cases, and market access - would be helpful in ensuring the UK can maintain its position as a global leader in Safety Tech across a number of technical themes and solutions.

Setting Standards & Guidance

Consultees noted that UK Safety Tech is home to a range of high-quality providers who want to show transparency in their approach to identifying and countering online harms. They felt there is a clear role for Government and a future online harms regulator to explore how standards can be set and independent auditing of solutions can take place. This should help to provide potential customers with an assurance that the solutions they procure meet a particular level of technical and ethical rigour.

“We should regulate the online Safety Tech industry – I want to be independently audited. You should check that I do have strong privacy, check my code of conduct, and that we put child safety first. Once I’ve achieved that – then promote me, and allow me to work with Government and industry.” - Established UK Safety Tech provider of age verification

Consultees noted that there are often complexities contained within navigating the legislative framework and in particular how to balance Safety Tech with privacy rights. This can make the landscape extremely complex and means that “getting a start-up off the ground in this space is not easy”. In this respect, shared guidance alongside legal and regulatory clarity would be particularly welcomed by the Safety Tech market.

Age Assurance

The Age Verification sub-sector has grown rapidly over the past three years, and applied lessons from similar tech sectors in organising itself as an industry. It includes firms such as Yoti, AgeChecked, and VeriMe – who continually seek to innovate solutions to making age verification more effective for users and businesses.

The founding members of the Age Verification Providers Association agreed a Code of Conduct to establish a common set of internationally applicable rules to provide a solid foundation for the sector’s reputation. They also worked with the British Standards Institution (BSI) to develop an industry standard for conducting age verification – PAS 1296:2018.

The sector is now working together to develop open-system protocols for interoperability of their solutions, and as the sector matures, an attribute exchange modelled on payment networks which will create a sustainable, open and competitive market in certified online age-assurance.

The British model is being studied closely by other jurisdictions, as far apart as Australia, Poland, New Zealand, France, Ireland and Germany with the sector setting the international standard and positioned to be a major UK export success story.

The UK is also home to innovations in age assurance, such as:

• Yoti’s Age Estimation AI, that can be used to estimate the age of users under 18 . For example, Yoti were recently featured in a BBC documentary called ‘nudes4sale’ where they were able to flag potentially under-age users publishing illegal content through social media platforms.
• TrustElevate’s verification of a parent-child relationship, age-checking of children and obtaining parental consent by conducting privacy-preserving checks against authoritative data sources.

NSPCC

The NSPCC is the UK’s leading children’s charity and works to prevent abuse and neglect. One of the NSPCC’s core strategic goals is to keep children safe from technology-facilitated abuse, though providing innovative resources and practical advice for parents and professionals, offering direct support to children, and campaigning for changes to legislation and industry practice to ensure children receive better design protections.

The NSPCC regularly publishes high quality, independent research about the risks that children face when using social networks and gaming platforms, and strives to provide thought leadership about how safety practices can be embedded into platform design, organisational culture and public policy. It sees the emergence of a thriving Safety Tech sector as being a crucial component of the move towards a Duty of Care, and to ensure tech firms can offer meaningful protections to children online.

It supports an emphasis on evidence-based interventions that can promote better long-term outcomes for children.

The NSPCC’s recent publications include ‘How Safe Are Our Children? 2019: An Overview of Data on Child Abuse Online’, and forthcoming research on the risks faced by children using social networks and gaming platforms.

7. International Market

7.1 Introduction

This section illustrates the performance of the UK Safety Tech sector in respect of other countries and sectors, to generate comparative illustrations of the potential for future growth.

7.2 International Market

The Safety Tech sector demonstrates a keen desire to export and work internationally, in full recognition that technologies and solutions developed within the UK can have global relevance.

“The DNA of our company means that (as a Safety Tech firm), we have to operate within an international market. The addressable market opportunity is global. The big tech firms are often based in the UK, but they’re not all UK headquartered firms. This means we need to sell internationally.” - Established UK provider of social harm intelligence and moderation services

“Good technology is exportable. The UK should want to protect children across the world.” - Large multinational Safety Tech provider

Export Focused

Within this research, we sought to identify companies that demonstrated an export presence (e.g. through sales, or office location) through data gathering and consultations.

We estimate that almost half (47%, 33 companies) of dedicated Safety Tech companies have an international presence. Given that an estimated 9.8% of all UK SMEs export, this demonstrates the comparable significance of exporting for UK Safety Tech firms.

The export sector is even more significant for larger Safety Tech firms: of the largest 20 firms by revenue, two-thirds (65%) have demonstrated some form of international or export activity. Key export markets include countries such as United States, Germany, Sweden, United Arab Emirates, Israel, and Australia.

UK Market Share

This research project is the first to analyse the size and scale of the domestic and international Safety Tech market. There is therefore little data to draw from regarding the UK’s proportion of global market share.

As a first step towards a benchmark, we have identified over 60 businesses that were based in other countries (not based in the UK) that would be likely to meet the Safety Tech definition in this report.

Our analysis of these businesses suggests that UK providers are likely to represent at least 25% of international market share for independent Safety Tech providers.

This figure, however, should be explored and tested as the Safety Tech sector matures internationally in the coming years and a clearer sense emerges of the breadth of international providers.

In addition, the UK is perceived as being uniquely influential on the global stage. Consultations suggest that the UK is globally recognised as having a leadership role with respect to policy and business activity.

Feedback was also highly positive from UK and international stakeholders ^{[footnote 11]} regarding the UK’s contribution and market share:

The UK has been raising the profile of online harms for several years, and is at the front end internationally.

The UK is the lighthouse for online safety. We’ve seen some of the first major steps taken by a country to regulate in the area of online harms.

The UK is a global leader. Initiatives like WeProtect and the IWF are seen as world leading.

The UK appears to be investing in law enforcement, and there’s no other country on the same level as the UK.

Stakeholders noted that several UK Safety Tech firms had developed their businesses through international engagement, and that support for exporters, as well as further developing the UK market as a place to do business would be highly beneficial.

Without being dismissive about the UK, my customers aren’t in the UK. They’re in the US, Germany, Australia etc. It would be helpful for government to help create the infrastructure to make it easier for me to sell in the UK.

In recent months, we have seen momentum regarding UK Safety Tech in the media and within government.

The map below sets out an overview of some key regions identified within consultations, including key policies and organisations identified by consultees.

International Safety Tech providers

Canada	Two Hat: Two Hat Security is dedicated to keeping online communities safe from harassment and abuse. Two Hat provides an AI powered content moderation platform that detects high-risk content for online games, social networks, messaging apps, and more.
United States	L1ght analyse and predict online toxicity and harmful content using data science techniques.
United States	ModSquad provides on-demand customer support, content moderation, social media, and community management services globally.
United States	NewsGuard rates more than 4,000 websites responsible for approximately 95% of all the news and information consumed and shared online in the US, UK, France, Germany and Italy. This informs users if a website is reliable as they browse online news, and helps to counter misinformation online. NewsGuard has an editorial and policy operation in the UK, and is listed by Ofcom as a useful tool for fact checking and debunking online.
United States	Thorn builds technology to defend children from sexual abuse. Its Spotlight platform is used to accelerate victim identification in juvenile sex trafficking cases. Its latest solution, Safer, helps tech companies proactively identify, remove, and report child sexual abuse material from their platforms.
Denmark	SafeonNet provides online reputation insurance and protection. This includes support with removing unwanted content online e.g. revenge pornography.
Netherlands	Deep Trace Labs: DeepTrace provides deepfakes detection solution designed to ensure the integrity of visual media.
Germany	Germany adopted its Network Enforcement Act (‘NetzDG’) in 2017. This law requires online platforms with more than two million registered users in Germany to remove ‘manifestly unlawful’ content – non-compliance risks a fine of up to €50 million.
Sweden	Besedo provides content moderation services, including profanity filters. AI powered nudity detection, and customised AI moderation models.
Sweden	NetClean are experts in detecting child sexual abuse material in business IT environments through using hashing technology.
Israel	Jiminy provides parents and guardians with the information they need to understand what their child is doing online and will provide an alert if children are subject to social issues, concerning content, or toxic device usage.
Australia	Australia established an eSafety Commissioner through its Enhancing Online Safety for Children Act in 2015. The eSafety Commissioner is responsible for promoting online safety for all Australians. It developed a Safety by Design initiative in 2018, which places user safety at the forefront of online service development — creating safer environments online.

8. Summary and Recommendations

8.1 Key Findings

In this report, we have defined the constituent parts of the UK Safety Tech market, measured its current economic and social value and illustrated its potential for future growth.

Defining ‘Safety Tech’

Safety Tech providers develop technology or solutions to facilitate safer online experiences, and protect users from harmful content, contact or conduct.

For example, these technologies:

• detect and remove known illegal imagery, relating to child sexual abuse or terrorism

• help moderators detect and address harmful or illegal user interactions, such as cyberbullying or grooming

• help platforms detect understand use of platforms by children

• work at device level to limit children’s exposure to harmful material; and

• identify potential sources of disinformation and promote verified facts

Scoping the UK Safety Tech market

Based on this definition, we have identified:

• A significant market - there are 70 dedicated Safety Tech businesses within the UK, generating £226m in annual revenues in 2019 and a current post-money valuation of £503m

• Rapid growth rates - the UK safety tech sector experienced an average estimated 35% annual growth rate since 2016, with leading firms growing at up to 90%

• Substantial increases in investment - the past four years have seen external investment in the sector increase more than eight-fold, to a record year in 2019 with £51m raised across 19 deals

• Employment hubs across the UK - Safety Tech businesses hire over 1,700 full-time equivalent (FTE) staff in the UK, with emerging regional centres of excellence in London, Leeds and the Midlands, Cambridge and Edinburgh; and

• A strong export sector - almost half (47%) of UK Safety Tech companies have an identifiable international presence, and UK companies comprise approximately 25% of global market value

Safety Tech providers also combine profit with purpose, and their products are likely to have made a positive difference to the lives of millions of internet users worldwide.

Towards a billion-pound sector

We have detected strong ongoing growth trends, which suggest the UK Safety Tech industry is set to capitalise on this early success. These include:

• A billion-pound sector, based on current trends - if current growth trends of 35% per annum continue, the sector could generate £1bn in annual revenues by the mid-2020s; and
• Emergence of ‘unicorn’ Safety Tech companies - there is a consensus that the UK is likely to see up to four Safety Tech unicorn companies (i.e. worth over $1bn US dollars) companies emerging by the mid-2020s

These growth trends are underpinned by: * A uniquely supportive ecosystem - UK strengths in online safety, emerging technology, and innovative regulation have created an environment in which Safety Tech is well-placed to flourish; and * A move towards regulation - the introduction of an online harms regulator is likely to see further interest in the use of technologies that help companies achieve compliance

Recommendations

Throughout this research, we have sought to understand how the UK Safety Tech sector may be better supported to grow sustainably, and the benefit as many users as possible.

Primary recommendations (to government) from those we spoke to in compiling this report cover the following areas:

1. A need to promote and increase awareness of the UK Safety Tech sector
2. Supporting Safety Tech firms to access the right forms of capital for growth
3. Getting the policy landscape right – with government providing leadership, guidance and appropriate legislation to address online harms
4. Enabling improved access to data, including promotion of data-sharing, privacy by design, and collaborative partnerships
5. Supporting innovation, cross-disciplinary research and development in Safety Tech

Theme 1: Promoting & Increasing Awareness

UK Safety Tech has a range of technical solutions to create safer online communities. However, demand-side awareness of these products is limited. Consultees highlighted the need for a sustained series of initiatives to promote and enhance awareness of Safety Tech among social media platforms, ISPs, schools, parents, governments and investors are supported wherever possible.

These could include regular cross-industry roundtables with Government and public sector stakeholders, to explore how to support the sector.

Recommendations (for industry and government):

( 1 ) Establish a UK Safety Tech Industry Association ^{[footnote 12]} to help create a consistent voice for the sector, and coordinate awareness-raising activity.

( 2 ) Raise awareness of Safety Tech. Work across sectors to showcase UK Safety Tech providers and create stronger networks to connect buyers and suppliers, and to drive adoption.

( 3 ) Grow safety tech exports, and work with the Department for International Trade (DIT) to review market opportunities and carry out trade missions.

Theme 2: Access to Capital

Investor confidence within UK Safety Tech has clearly grown in recent years; however, the volume and scale of investment could grow further in the years ahead.

There is no ‘one-size-fits-all’ approach to investment in Safety Tech. Some businesses have been able to secure investment aligned to positioning their offer as ‘RegTech’, whilst others have accessed socially-driven investment to meet their objectives.

Several stakeholders highlighted the value of grants or match funding to help them undertake exploratory projects. There are also several Safety Tech businesses that have identified the UK’s EIS tax reliefs as particularly useful in supporting high-worth individuals invest in their businesses.

Public procurement was also cited as a critical component of helping the sector to grow, with respect to both ease of access and increased opportunities to work with the public sector and law enforcement.

Recommendations

( 4 ) Review the financial and practical support given to UK Safety Tech organisations, to ensure they have sufficient access to investment.

As part of this, investigate the use of grants and challenge funds to drive start-ups, scale-up and innovation, and opportunities to commercialise work coming from the academic sector. This might include exploring mechanisms to support new Safety Tech academic start-ups to commercialise ideas, or initiatives to encourage further investment in UK Safety Tech.

( 5 ) Update public procurement guidelines, to ensure that public sector organisations are making sufficient use of Safety Tech to protect their own systems and members of the public.

Theme 3: Policy & Standards Landscape

The UK Safety Tech sector perceives that there has been considerable policy activity in recent months, and there is a real need for this momentum to continue. There is considerable desire among stakeholders for Government to allocate resources for departments and the regulator (most likely Ofcom) accordingly to invest in projects. These also need to span industry (reflecting match funding), academia, and charities.

“A regulator will definitely increase growth potential, but you need to start actively creating solutions. It wouldn’t take much resource to do some exciting things e.g. co-fund online safety challenge funds or accelerators.” – UK Safety Tech Think-tank

Recommendations

( 6 ) Explore opportunities for the UK Safety Tech market to address regulatory compliance needs, and ways through which online harms regulation can best support innovation.

( 7 ) Design and implement a sector-wide Safety Tech strategy, including mechanisms to help cross-sector collaboration - for example, regular government / industry roundtables or innovation networks.

( 8 ) Promote development and use of consistent and robust standards: The UK has already developed best practice standards in online safety. These should be promoted wherever possible, and new standards developed where appropriate.

Theme 4: Access to Data

Artificial Intelligence (AI) tools play an essential role in helping companies detect and respond to online harms at scale. The effectiveness of these systems rely on the quality of the data on which they are built and tested.

However, although the UK has a number of key data assets that can be drawn upon to grow Safety Tech products, such as Home Office Child Abuse Image Database (CAID), many consultees have described challenges in accessing high-quality data illustrating online harms. For example, current standards do not support interoperability, and mechanisms for safe and secure data-sharing are scarce. This is limiting the growth of a competitive market in AI-driven Safety Tech solutions.

Recommendations

( 9 ) Government should explore a mapping exercise of the online harms data landscape, including a review of current access routes to relevant data e.g. use of sandbox environments, or engagement with existing Government datasets. This might include prototyping approaches to deliver greater interoperability through common data standards and vehicles for data-sharing, and explore potential for implementation of approaches with industry partners.

Theme 5: Supporting Innovation, Research and Development

Within recent years, there has been considerable growth in the UK’s Safety Tech sector. However, this growth has been supported by progress made within UK academia, and investment in R&D.

For example, some of the UK success stories set out within this report have been academic spin-outs, drawing upon world-class research and commercialising this IP into viable and successful businesses.

Consultees noted that they would welcome research councils (e.g. UKRI) establishing a bespoke research agenda for Safety Tech in the UK. There is a distinct need to establish ‘Safety Tech’ research opportunities within the UK, rather than include this as a cyber security component.

Further supporting distinct programmes of research, as well as enabling funding for novel or ad-hoc projects to tackle online harms would also help to increase standards and transparency of approaches within the sector.

Recommendations

( 10 ) Explore opportunities for enhanced academic research in UK Safety Tech with UKRI. Specific research calls should explore technical (and other) approaches to addressing online harms, including testing products in the marketplace, and evaluation of ‘what works’.

Appendix A: Research Team

Perspective Economics (www.perspectiveeconomics.com

Perspective Economics provides independent economic analysis and advisory services. They use modern methodologies and analytics to gather and interpret evidence; providing insight and clarity on important economic, business and social issues. They have significant experience in conducting sector studies across the UK, exploring the economic contribution and labour markets of emerging industries and sectors such as cyber security, HealthTech, AI, and SafetyTech. The study lead for this research was Sam Donaldson (Director).

Academic Advisors

Mary Aiken PhD is a Professor in the Department of Business and Law at the University of East London (UEL), Adjunct Professor at the Geary Institute for Public Policy University College Dublin (UCD), Associate Director of the Online Harms and Cyber Crime Unit at UEL, Academic Advisor to Europol’s European Cyber Crime Centre (EC3), Fellow of the Royal Society of Medicine, member of the Medico-Legal Society of Ireland, International Affiliate Member of the American Psychological Association (APA) and Fellow of the Society for Chartered IT Professionals.

Julia Davidson, PhD is a Professor of Criminology in the Department of Business and Law at the University of East London (UEL), Director of the Online Harms and Cyber Crime Unit at UEL, Chair of the Evidence Group for the UK Council for Child Internet Safety (UKCCIS), has served as a member of the UK Inquiry into institutional child abuse Academic Advisory Board, and is now Chair of the Ethics Committee, Academic Advisor to Europol’s European Cyber Crime Centre (EC3), Honorary Research Fellow at Royal Holloway University of London, Fellow of the UK Higher Education Academy and former Adjunct Professor in the School of Law at QUT, Australia.

Appendix B: Methodology

This section sets out our research methodology for identifying, including/excluding online safety technology organisations, and capturing their respective economic contribution.

Stage 1: Desk Research and Data Collation

Using a Grounded Theory approach, the team reviewed over 80 pieces of academic literature, sector overviews, and grey literature. Some of the core literature used within this research is included within the bibliography.

We used a Grounded Theory ^{[footnote 13]} approach, whereby data and evidence was collected and analysed systematically to identify categories and emerging findings that could be tested with industry and policy-makers. For example, this meant that we were able to rapidly:

• identify Safety Tech providers within the UK that offered a range of different solutions to various problems relating to online harms; and
• categorise these based upon variables such as the technologies and approaches undertake to address these e.g. content moderation, or user protection

Extensive data collation means that we were able to develop a long-list of potential Safety Tech businesses within the UK and internationally, to identify the characteristics of Safety Tech organisations. This was subsequently tested and refined using company descriptions, and throughout workshops and engagement with industry to ensure that our short-list of Safety Tech firms was aligned to an agreed definition.

Some of the key sources utilised to create a long list of online safety providers included:

• Internal Monitoring and Market Intelligence: Upon commencement of this project, DCMS provided an overview of known providers within the market that they had either engaged with or were aware of their work. Further, some alignment between firms known to DCMS (e.g. through cyber security cohorts) with an online safety focus were flagged by Perspective Economics. However, it is worth noting that this monitoring included firms with a global (not only UK) reach, in addition to known products and services offered aligned to online safety by larger tech firms e.g. Microsoft PhotoDNA.
• Internet Watch Foundation: The Internet Watch Foundation has over 140 commercial members, including some of the world’s largest tech companies to smaller filtering companies. Each member (depending on membership tier) pays between £1,085 and £81,185 per annum to support the work of the IWF. To inform this analysis, we reviewed the IWF membership within the long list, to help identify where firms are either providers, users, or seek to fund the development of Safety Tech products and services.
• Bureau van Dijk FAME: Following an online safety and online harms literature review, our team identified over 200 search terms (aligned to the Safety Tech taxonomy set out within the report). These were used as inputs into Bureau van Dijk FAME (a database of UK registered organisations) to identify firms potentially relevant to the online safety taxonomy (e.g. providers of content filtering solutions). This was further augmented with web scraping to identify other providers or networks aligned to these products and services.
• Other Market Intelligence: Our team also conducted a desk review of relevant market intelligence (e.g. reports commissioned by DCMS, charities and tech bodies), representative bodies (e.g. associate members of the UK Council for Internet Safety (UKCIS)) to identify other providers of online safety solutions (for addition to the long-list).

Through the long-listing approach, we identified 326 organisations through this process.

Following the development of the long list, we sought to understand these organisations in further detail. Each of these organisations were reviewed to identify metrics including (but not limited to):

• company name, registered number, company status, and date of incorporation
• registered and trading locations
• company website and contact details
• trading description of company activities related to online safety technology; and
• estimated company size

Subsequently, there were several steps to be taken to yield a final working short-list for purposes of analysis. These included:

• Removing duplicates: Many of the organisations identified shared similar sources, or the registered company and brand name were different within the initial sources. On this basis, we removed duplicate entries within the long list;
• Filtering by organisation type: As this exercise is focused upon identifying the size and scale of the commercial Safety Tech market, we excluded universities and charities; however, social enterprises / tech for good firms with identifiable revenues or commercial activity were retained for analysis
• Filtering by location: As stated, this exercise explores the UK market, and therefore organisations with no clear presence in the UK (typically registered status) were removed
• Company Status: Where businesses were since dissolved or inactive, these were removed from the analysis; and
• Removing organisations not aligned to the taxonomy: Finally, subsequent to the development of the sector taxonomy, organisations viewed to be not aligned with the taxonomy (i.e. trading descriptions did not sufficiently demonstrate an offering of online safety products, services, or expertise) were removed

This process yielded the 70 Safety Tech providers based within the UK, and aligned to the research brief for analysis within this report.

Stage 2: Definition and Market Scoping

Alongside the desk research and data collation, the research team also developed a working definition for what constitutes ‘Safety Tech’. This also informed the development of a ‘Safety Tech’ taxonomy within this research, and the short-listing of Safety Tech firms i.e. those that provide a product or service aligned to the categories.

As set out within the report (Sector Taxonomy), we identified a range of activities undertaken by Safety Tech providers. These included:

System-Wide Level Factors:

This refers to organisations involved at the highest levels, often working closely with law enforcement, to trace, locate and remove (or help to facilitate the removal) of illegal content online.

System-Wide Governance:

These organisations help to identify and tackle some of the internet’s most harmful content e.g. child sexual abuse and exploitation and terrorist content. This can be achieved through:

• Working closely with law enforcement to assist with investigative capabilities e.g. use of forensic science to scan for known illegal content using MD5 hashes
• Maintaining and providing access to technology aimed at preventing the upload, and facilitating the removal of illegal content e.g. the IWF’s Hash List (with Microsoft PhotoDNA); and
• Combating abuse or threats with automated content analysis and artificial intelligence e.g. automated detection of terrorist content, including previously unseen material

Platform Level Factors:

This refers to organisations that are involved in making online services safer and typically work at the platform level i.e. work alongside social media, gaming, and content providers to improve safety and behaviour within their platforms.

These have been segmented into the following sub-categories:

(1) Platform Governance:

These organisations are focused upon helping providers of online content to govern their offering with respect to illegal content. Whilst there is some overlap with ‘System Governance’, this is more focused upon organisations that help to tackle issues such as:

• Embedding prevention mechanisms e.g. using machine learning to prohibit the production of indecent underage imagery on social media platforms
• Identifying and blocking harmful images and videos in real-time; and
• Identifying child abuse or grooming in conversations

(2) Platform Moderation & Monitoring:

These organisations also help providers of online content to monitor and moderate behaviour and content posted within their platforms. This is typically focused upon reducing harmful content or behaviour e.g. offensive language, bullying, or toxic content.

This can include:

• Moderation and monitoring of content e.g. pre-moderation or post-moderation of content, undertaken by automated content analysis and / or humans
• Chat moderation e.g. identifying and removing users subject to language or words used; and
• Behavioural Monitoring e.g. identifying good and bad behaviour, typically using Natural Language Processing within online communities

(3) Age Orientated Online Safety:

These organisations seek to support online content providers in ensuring that their platforms are either age-appropriate and increase the privacy of children online (e.g. compliant with GDPR-K, or that the content and access requirements are suitable should the website or app be targeted at under-18s, thereby ensuring ‘safety by design’), or provide age-assurance services (i.e. help companies to validate and confirm that only particular age groups can access particular content).

Endpoint Level Factors:

This refers to organisations that provide products or services that help to ensure that the device being utilised by the end-user is suitably secure with respect to online safety. This focuses upon online safety solutions (i.e. ensuring that the user’s risks with respect to content, conduct, and contact are reduced). It does not include endpoint protection from viruses, malware, or adware – which are covered by ‘cyber security’.

User protection at the endpoint level can be segmented into two main categories.

(1) User initiated protection (user, parental or device-based):

This includes organisations that provide products or services that can be installed on devices to help secure the end-user from online harms (typically a parent or guardian installing on behalf of a child). The underlying ambition is to create a safer online experience for the user e.g. through safeguarding assistants, oversight of social media content, or through monitoring of a child’s digital or online behaviour and interaction with other users. Where deployed, these solutions can help to prevent issues relating to sexting, grooming, bullying, abuse, or aggression.

(2) Network filtering:

This includes organisations involved in providing products or services that actively filter content (e.g. through white-listing or black-listing, or through actively blocking content perceived to be harmful or illegal). This can often include solutions provided to schools or home users to filter content for users.

Information Environment Level Factors:

This refers to organisations that actively detect and disrupt false, misleading and / or harmful narratives. This includes tackling misinformation and disinformation through the provision of fact checking and disinformation research and disruption. Organisations within this space seek to ensure citizen information accuracy and facilitate trust in the information environment and wider society.

Other

Online Safety Professional Services: This includes organisations typically involved in supporting the design, implementation and testing of online safety through the provision of compliance services, research, frameworks and methodologies for auditing, evaluating or mitigating potential harms, and help to enable the development of safer online communities.

Support: Further, this analysis has also sought to identify organisations involved in supporting the development and scaling of online safety products and services but do so in an advocacy capacity e.g. charities.

Stage 3: Workshops

We held four stakeholder workshops throughout this research including representatives from industry, academia, charities and policy-makers to test and agree the scope of this research. These typically involved interactive discussions from a wide range of stakeholders to discuss, test and refine the research findings, including:

• Extensive feedback regarding a ‘Safety Tech’ working definition and taxonomy
• Review of industry members for inclusion in the analysis
• Horizon scanning – to identify particularly important developments within the sector and those impacted by the issue of online harms
• Testing the market assumptions (including size estimates, growth potential, investment and exports); and
• Sharing real-world experience of the opportunities and challenges for the sector

Stage 4: One-to-One Consultations

This research is informed by 50 direct interview consultations with a range of Safety Tech stakeholders (in the UK and internationally), covering the views of industry, academia, charities and policy-makers. This included gathering feedback about the respective strengths, challenges and opportunities for the sector.

Each consultation was focused upon four key themes where appropriate including:

• The extent of their organisation’s involvement in the sector (i.e. what products and services they offered, and an overview of their organisation’s growth to date)
• Views on market trends, drivers for demand, and investment in the sector
• Views on the extent of products, services and innovation in the marketplace; and
• Views on what government’s role could or should be in helping to support growth in the sector, and increase uptake of Safety Tech solutions

In order to gather robust qualitative findings, these consultations were anonymised – to enable stakeholders to speak openly about their views on market opportunities, barriers, and support initiatives.

Stage 5: Market Analysis

The UK’s Safety Tech sector does not have a formal Standard Industrial Classification (SIC) code, and therefore the figures set out within this report are market estimates conducted by Perspective Economics.

We have drawn upon the following sources to gather the required information to inform these estimates, including:

Bureau van Dijk’s FAME platform:

This provides relevant data on all UK companies, including location, trade description, revenues, profits, assets, remuneration, depreciation and amortisation, employment, and SIC codes.

This enables estimation of revenue, employment and Gross Value Added (GVA) at the sector level. However, this does not include these metrics for several micro and small companies that are not typically required to report full financial information to Companies House. For these companies, we have estimated the financial performance of micro and small firms within the sector, informed through direct consultations, and employment headcount figures.

Beauhurst:

Beauhurst (www.beauhurst.com) provides detail on approximately 50,000 high growth companies in the UK, including investment data, grants, and participation in Government accelerator schemes (including Innovate UK activity). We use this to analyse the investments, grants, loans, stake taken, funders, size, scale, employment and valuation of online safety firms.

Company Level Data:

All organisations identified have an online presence (i.e. a website). We have sourced the trading description from each organisation, typically within the ‘About Us’ section. This means that each description is ‘as-described’ at the organisational level and used to segment (at this stage) where each organisation’s products or services align to each section of the online safety technology taxonomy. It should be noted that this will be revisited during consultations, as often organisations may offer products or services further to those posted on their trading website.

Appendix C: Bibliography

This sets out the literature sources used to inform the development of the online safety technology definition, taxonomy, and the long list / short-listing of firms in scope.

Baines, V (2019) Member state responses to prevent and combat online child sexual exploitation and abuse, Council of Europe.

Briggs, L (2018) Report of the Statutory Review of the Enhancing Online Safety Act 2015 and the Review of Schedules 5 and 7 to the Broadcasting Services Act 1992 (Online Content Scheme) Commonwealth of Australia.

Cambridge Consultants (2019) Use of AI in Online Content Moderation, Ofcom.

Centre for the Analysis of Social Media DEMOS (2017) Technology Briefing Series Briefing 1 Online Child Sexual Abuse Imagery.

Davidson, J., Livingstone, S., Jenkins, S., Gekoski, A., Choak, C., Ike, T., and Phillips, K. (2019) Adult Online Hate, Harassment and Abuse: A Rapid Evidence Review, UK Council of Internet Safety.

Department for Digital, Culture, Media & Sport (2018) Rapid Evidence Assessment: The Prevalence and Impact of Online Trolling, Centre for Strategy and Evaluation Services.

Department for Digital, Culture, Media, & Sport (2019) Online Harms White Paper, UK Government.

eSafety Commissioner (2019) Safety by Design Overview, Australian Government.

Green, A., Wilkins, C., Wyld, G., (2019) Keeping Children Safe Online, NPC, Nominet.

Green, L. R. (2010). Internet savvy? Children and online risk. Proceedings of Communications Policy and Research Forum 2010. (pp. 226-235).

Sydney, NSW. Network Insight Institute Houses of Parliament Parliamentary Office of Science and Technology (2019) Online Safety Education.

Hugill, J., and Puvinathan, R., (2019) How To Deliver Innovation Through Public Procurement, Public.

ICF (2019) Research into Online Platforms’ Operating Models and Management of Online Harms, UK Government.

Information Commissioner’s Office (2019) Age Appropriate Design: A Code of Practice for Online Services Consultation Document.

Internet Safety Technical Task Force (2008) Final Report of the Internet Safety Technical Task Force to the Multi-State Working Group on Social Networking of State Attorneys General of the United States

Internet Watch Foundation (2018) Annual Report.

Jigsaw Research (2019) Internet Users’ Concerns about and experience of potential online harms, Ofcom, Information Commissioner’s Office.

Livingstone, S and Haddon, L (2009) Introduction. In: Livingstone, S and Haddon, L, (eds.) Kids online: opportunities and risks for children. The Policy Press, Bristol, UK, pp. 1-6.

Melford, C., and Fagan, C. (2019) Cutting the Funding of Disinformation: The Ad-Tech Solution, Global Disinformation Index.

Nash, V., O’Connell, R., Zevenbergen, B., Mishkin, A., (2013) Effective age verification techniques: Lessons to be learnt from the online gambling industry, University of Oxford.

National Children’s Bureau Northern Ireland (NCB NI) on behalf of the Safeguarding Board for Northern Ireland (SBNI): An exploration of e-safety messages to young people, parents and practitioners in Northern Ireland.

O’Neill, B., Aiken, M.P., Caffrey, M., Carthy, J., Lupton, R., Lynch, A. & O’Sullivan, K. (2014). Report of the Internet Content Governance Advisory Group. Department of Communications, Energy and Natural Resources, Ireland.

Ofcom (2018) Addressing harmful online content A perspective from broadcasting and on-demand standards regulation.

WePROTECT Global Alliance (2019) Global Threat Assessment 2019: Working together to end the sexual exploitation of children online, (http://www.weprotect.org)

PA Consulting (2019) A Tangled Web: Rethinking the approach to online CSEA – A proposal to tackle the growing threat of online child sexual exploitation and abuse (http://www.paconsulting.com)

Panel for the Future of Science and Technology (2019) Automated tackling of disinformation. European Parliament.

PwC (2019) Kids digital media report, Super Awesome.

Roberts, S. (2016) Commercial Content Moderation: Digital Laborers’ Dirty Work, FIMS Publications.

Tech Nation (2019) Tech for Social Good.

Technical Working Group (2018) Child Dignity Alliance: Technical Working Group Technology Inventory, Child Dignity in the Digital World.

techUK (2019) Digital Strategy shows UK a world leader in Child Internet Safety

Two Hat Security (2019) There’s no single solution to keeping children safer online, but there is hope

UK Council for Child Internet Safety (2015) Child Safety Online A Practical Guide for Providers of Social Media and Interactive Services.  

1. See factsheet on Child Sexual Exploitation and Abuse ↩

2. The UK Online Safety Tech Industry Association (OSTIA, www.ostia.org.uk was founded in April 2020, shortly after the research informing this report was completed. ↩

3. DeMarco, J., Cheevers, C., Davidson, J., Bogaerts, S., Pace, U., Aiken, M.P., Caretti, V., Schimmenti, A., & Bifulco, A. (2017) Digital dangers and cyber-victimisation: a study of European adolescent online risky behaviour for sexual exploitation. Clinical Neuropsychiatry, 14 (1). pp. 104-112. ISSN 1724-4935 ↩

4. CAGR: Compound Annual Growth Rate – 35% ↩

5. https://technation.io/news/uk-tech-expands-at-faster-rate-than-the-wider-economy/ ↩

6. Where companies have not secured external investment, or have sought not to disclose investments – it is not possible at this stage to determine their market valuation, which requires detailed firm and market appraisal. ↩

7. https://www.funds-europe.com/news/interest-grows-in-esg-among-uk-investors ↩

8. Quotes 1 and 4 from an Academic Lead for Online Safety (UK university), Quote 2 from a UK AI / SafetyTech start-up, Quote 3 from aUK Safety Tech spin-out. ↩

9. Range of feedback from International Safety Tech Providers and Policy-Makers (United States, Canada, Australia and Sweden) ↩

10. The UK Online Safety Tech Industry Association (OSTIA, www.ostia.org.uk was founded in April 2020, shortly after the research informing this report was completed. ↩

11. Glaser BG, and Strauss AL (1967) ‘The discovery of grounded theory: strategies for qualitative research’ ↩