Research and analysis

Thematic review and gap analysis on AI security

Published 10 July 2026

1. Executive Summary

The Department for Science, Innovation and Technology commissioned Lancaster University to conduct a thematic review and gap analysis covering peer-reviewed research on the security of AI from the last five years (January 2021 to January 2026). This report was sought to provide a detailed overview of the state of research in this field. A comprehensive review of the literature was undertaken using a PRISMA approach which involved identifying relevant sources using keyword search terms within two databases, Scopus and Web of Science. A sifting process was then undertaken followed by a screening process which resulted in 9,109 publications being identified. The study used data-driven techniques including filtering and semantic matching to map the research into themes as part of analysing the results.

The review was able to group the publications under 12 themes. These are:

• Training Time Security

• Data and Privacy Risks

• Alignment

• Supply Chain Vulnerabilities

• Inference Time Security

• End of Lifecycle and Disposal Risks

• System Infrastructure Security

• Security Risks Arising from Failure to Track Assets

• Autonomous Agent Security

• Security Risks Arising from End User

• Model and System Design Security

• Security Governance and Regulation

The research also found various gaps in the literature, with 5 prevalent areas where more research is needed: 

• Assurance and verification methods for the integrity of AI system data, such as training data or model weights.

• Techniques to track and verify the provenance of third-party AI models.

• Bridging the gap between the AI model attack surface, and the traditional IT attack surface of the infrastructure underpinning it, for generative and agentic-AI systems.

• Understanding how the end-user can pose a security risk to an AI system through their actions, and how to mitigate this.

• How to safely decommission and dispose of models, particularly for frontier AI systems.

The review also found significant gaps on the cyber security of Agentic-AI systems, specifically the security of the agent themselves, the tools they use to interact with their environment, and inter-agent communication and operation. The authors anticipate that this area will gain more focus as deployment increases.

Please note this report presents independent research commissioned by the Department for Science, Innovation and Technology. The views expressed do not represent UK government policy and the report’s findings are derived from research led by Lancaster University.

2. Introduction

Artificial Intelligence lies at the heart of growth strategies both nationally, and internationally, over the next 5 years. To ensure this goal is achieved, it is crucial that the AI systems underpinning this are secure. In recent years ‘AI Security’ has become an active sub-domain of cyber security research. Whilst publication in this field has continued at pace, there have been limited efforts to objectively understand which aspects of AI security are well studied, which ought to be studied, or where the gaps are. There is a need to understand this so that ‘blind spots’ in research and investment into AI security can be addressed.

The Department for Science, Innovation and Technology therefore commissioned Lancaster University to conduct a thematic review and gap analysis covering peer-reviewed research on AI security from the last five years (from January 2021 to January 2026). This report sought to provide a detailed overview of the state of research in this field.

3. Scope

All forms of AI were in scope of this study.^{[footnote 1]} This includes machine, deep, and reinforcement learning based systems, computer vision models, generative AI (including LLMs and Foundation Models), and concepts such as Agentic-AI. The review only examined English-language sources from January 2021 to January 2026. Additionally, only peer-reviewed sources were included. This means that reports published on company, or government websites which had not gone through a formal quality assurance mechanism were not included.

The report initially sought to identify any source on AI security to ensure no publications were missed before then focusing on those related to the cyber security of AI. This report has taken a broad view of the definition of the ‘cyber security of AI’. This can include but is not limited to: the cyber risks and vulnerabilities of the AI models themselves, the systems that underpin such models, and the security risks that arise in these systems as a consequence of end-use. This approach aligns with the ETSI standard EN 304 223 (ETSI 2025), and ISO/IEC 22989 (ISO/IEC 2022) in that it considers security risks and vulnerabilities that arise over each stage of the system lifecycle: design, development, deployment, maintenance, and end of life. This means that topics linked to AI safety as well as harms arising from the misuse of AI systems, for example to create deep-faked content or to perform cyber attacks on a target, were out of scope.

4. Methodology

The first phase of this project focused on determining the most robust approach for identifying sources on the security of AI. The authors decided to use two databases: Scopus and Web of Science because they are considered the most comprehensive repositories for accessing sources.  The second phase of the study focused on identifying all relevant sources in scope. Each of the databases were searched for reports whose titles, abstracts and/or author keywords matched a list of key words and phrases associated with AI Security.

The search terms are included in this report in Appendix A, and the Lancaster University authors will provide the source code on request.

The literature review had an additional inclusion criterion that reports had to have been published by a journal or conference proceeding from a list of top journals and conferences in AI, machine learning and cyber security.  This list was determined using H5 index and manually inspected to ensure a balance across these areas. H5 was used as it gives the most cited publication sources over the last 5 years, and this review covers literature from 2021-2025. Limiting the search results to reports published by journals and conferences in this list was to ensure that reports retrieved were relevant to the aims of this report. This is further discussed in the Limitations section. Finally, reports were restricted to those published from 2021 onwards and English language only.

Figure 1 gives an overview of this search process. The searches were carried out on Scopus on March 17, 2026 and Web of Science on January 22, 2026, returning 13,847 and 1,931 results, respectively. The databases were accessed via Lancaster University Library. After de-duplicating and quality control, 9,196 papers remain. The study then further sifted to remove student abstracts, introductions to volumes of proceedings, and papers out of scope, such as those about cryptocurrency, distributed ledger technology or Bluetooth. This left 9,109 reports included in the study.

Figure 1: Overview of Literature Search

Two of the charts in the exploratory data analysis section of this report are based on labels assigned by a Large Language Model.  To label papers in a dataset of this scale the authors used ChatGPT-5.4 mini. This model was chosen as it is a frontier LLM designed for broad general use cases. Firstly, it is used to assign an AI system to each of the papers in the study, based on the title, abstract and author keywords (if available). Secondly, it was used to assign a sector for author affiliations, to visualise the spread of affiliations through, for example, the academic and industry sectors. Further details are included in the discussion of each chart.

The authors have conducted the study so that it can be reproduced by any entity in the future. The source code will also provide additional information on how the methodology was implemented. As noted above, the source code can be provided by Lancaster University. Please contact AIcybersecurity@dsit.gov.uk who will pass on this request.

The report used specific data science techniques to identify the research themes and gaps from the literature. The technique, known as semantic matching,  calculated how well each paper’s title, abstract and author key words aligned with the descriptions of the themes and some additional key words and phrases, based on their meaning, via cosine similarity. ^{[footnote 2]} Each paper was assigned a score based on how well it was judged by the model to match with each theme. It was then assigned the theme for which it had the highest score. The method of semantic matching was judged to be more powerful than using key word matches as it avoids missing papers that align well with a theme but do not use specific pre-defined key words.^{[footnote 3]} The semantic matching threshold was verified manually by examining random samples of papers that met the threshold of matching with a primary theme.  Further details are in the code implementation.

Computing in this report, unless otherwise stated, was carried out in R (R Core Team, 2019). Data plots were created using ggplot2 (Wickham et al, 2019).  

4.1 Limitations

The limitations on the scope of this report are largely down to restricting the search for research reports to Scopus and Web of Science, as well as filtering the reports retrieved by publishing venue. This is an important restriction on the search to ensure relevance and the high quality of the research retained by the search. For example, a search on Scopus without restricting by publication, date or language of publication carried out on March 18, 2026 returned over 137,000 reports. This would be too large to handle within the time constraints, and would also include a lot of irrelevant material. 

Given the size of the study’s data set, it has not been possible to manually sift it. Data science techniques, such as cleaning and matching of venue titles, extracting random samples for manual review, and exclusion based on key words or phrases appearing in the title, abstract or author key words, have been employed to sift the data and ensure that papers included are relevant. More details are available in the code (available from the Lancaster University authors).

The counts of funding institution were compiled by string counting, with the top 50 entries checked manually for quality control.

In instances where data have been labelled using either a data science technique such as semantic embedding, or using an LLM, a sample has been manually checked. This is to verify accuracy, without having to perform an exhaustive check given the size of the data being dealt with.

Two limitations of the keyword gap analysis can be identified. The first is that the authors of the papers in the dataset themselves identify keywords, and so whilst these labels are likely to be indicative of what the research addresses, and in many cases have been peer reviewed, there is no objective guarantee. Secondly, there were 2,296 papers that were well-aligned, according to the theme matching method described in the methodology, and had no author keywords present in their meta-data. These were excluded from this analysis.

5. Report Findings

Research Themes

This study identified 12 research themes from the 9,109 sources. The themes have been ordered from most to least coverage across the 9,109 sources returned in this review, and are given below. After this the report presents a per-theme analysis giving a definition of each theme and a short summary of the most and least covered topics in each.

• Training Time Security

• Data and Privacy Risks

• Alignment

• Supply Chain Vulnerabilities

• Inference Time Security

• End of Lifecycle and Disposal Risks

• System Infrastructure Security

• Security Risks Arising from Failure to Track Assets

• Autonomous Agent Security

• Security Risks Arising from End User

• Model and System Design Security

• Security Governance and Regulation

Each of these themes can be mapped across existing global standards, particularly ETSI EN 304 223, on the cyber security of AI systems as illustrated in the Figure below.

Figure 2: Mapping of themes against ETSI EN 304 223

Training Time Security: This theme refers to the attempts of an adversary to compromise a model during its training phase. During training, the model/system learns its intended response to a given input. An adversary can then interfere with this training process to subvert the model behaviour, or insert a vulnerability into the system to exploit when the system is deployed. There are a range of different training time threats that an AI system is susceptible to, for example data poisoning, with attackers using techniques such as label flipping, or the introduction of harmful data, to alter model behaviour.

This is a well-covered theme, with 2101 papers well-aligned to this area. A significant number of papers focus on adversarial attack or defense (27%), and several more focus on specific types of attack. Given that they pose two of the most common types of training time security exploit, it is no surprise that backdoor attacks (13%) and poisoning attacks (7%) are common keywords in this area. What is interesting, however, is that this may be with good reason. For example, there are 21 different types of poisoning attack identified in this theme, illustrating why Training Time Security is both an interesting and important security research area, and should continue to receive attention.

That said, if there was one place where a gap was to be identified it would relate to the development of formal methods that ensure the data used to train the models has some level of formal guarantee that it is safe. This is a gap in the literature, with few papers considering this (<1%). At a somewhat higher level, this can be described as training data integrity, with data integrity or some variant again not a well-covered keyword (1% of papers).

Data and Privacy Risks: This theme refers to the risk of an adversary extracting sensitive or personally identifying information (PII) from an AI system. For example, an adversary may attempt to extract private user data from the model’s outputs; or information about the AI system itself through model inversion, extraction, or distillation attacks. These threats are harmful because they enable an adversary to ‘learn’ the properties of an AI system without training their own.

There are 1313 papers in this area, and a significant portion (21%) of papers in this topic refer explicitly to the security of AI systems in the context of privacy. There is also emphasis on papers that propose methods to enhance, or secure, the privacy of these systems (16%). Moreover, significant attention is given to specific types of attack that interfere with the data security of these systems. Papers focus on membership inference attacks (7%), and backdoor attacks (6%) and their interaction with data security.

Similar levels of coverage are exhibited for other common types of attacks against these systems. That said, fewer papers explicitly keyword the leakage of data (2%), showing that the focus is more on attacks to overcome, or methods to enhance, privacy security rather than the impacts of a successful exploit.

Finally, another area that has received little explicit coverage via author keywords is related to data security and verifiability (2% of papers). This is a potential gap because one does not only want to identify how data security can be breached, or how to better protect it, but ideally offer some level of guarantees on data integrity. This is therefore a potential gap where low-level technical research would be beneficial.

Alignment: This theme refers to security issues arising where an AI system and its operations are no longer in line with expected human intentions and values (Ji et al. 2025). Alignment has received attention in recent years with respect to AI systems based upon language models. Whilst it is not a risk that necessarily affects AI systems in general, and arguably it is linked to the safe use of AI rather than the security of the system itself, it is argued that an alignment failure can create an insider threat that will, in some circumstances, pose a security risk. As such, this is a theme worth including.

There are 200 papers assigned to this theme, and many of these cover the effect of malicious interactions on alignment. For example, several consider adversarial behaviour in general (9%), and this expands when specific attack vectors such as backdoors or injection are included (14%). What is less well studied, is how alignment relates to data risks. Indeed, few paper (1%) in this theme consider the interplay between data security and an aligned, or not, AI system. This presents a potential avenue for further research.

Supply Chain Vulnerabilities: This theme refers to how many AI systems rely upon third party components, libraries, or open-source models, and this reliance introduces a new component to the attack surface. For instance, all a threat actor needs to do is publish a malicious model, or one containing a backdoor, to platforms such as Hugging Face and then wait for an end-user to integrate them. This builds on top of traditional threat vectors for deployed machine and deep learning systems such as serialisation or dependency confusion attacks, whereby an adversary can use a single dependency to corrupt the entire AI system.

Of the 198 papers assigned to this theme, a well-studied area (13% of papers) is how adversarial behaviour can lead to users deploying models containing existing vulnerabilities, for example backdoors or poisoned data. This risk is becoming more exploitable due to the growing usage of repositories such as Hugging Face, although this is not covered in depth (2% of papers). This trend is repeated for papers understanding the provenance of third party models more generally (3% of papers), or for those considering the transfer of existing models into new systems (2%). As such, whilst supply chain security is not viewed as a gap in research overall, there are clear opportunities for further research in this area.

Inference Time Security: This theme refers to attacks that target deployed models and are designed to manipulate outputs, override guardrails, or impact on model/system performance at inference time. Examples of attacks range from adversarial attacks on computer vision or ML systems through to prompt injections and jailbreaks targeting large language models.

There are 212 well-aligned papers in this theme area. The overarching themes by keywords are connected to a type of malicious attack that can target an AI system (18% of papers). Subclasses of attack that are also covered in depth include model inversion or extraction attacks (6%), membership inference (4%), and LLM specific attacks such as jailbreaks or prompt injection (4%). One less studied element of malicious attacks, however, is the transferability of attacks from one model to another (less than 1% of papers).

Given how well covered attacks are at inference time, it would be easy to suggest that this theme has been well covered and that there are not gaps. It is argued that the opposite is true. The fact the author keywords for this theme are a blend of variants of AI systems, and attacks that target them, illustrates what a rich target for an adversary an AI system poses at inference time. This means that, as new AI systems are developed, there is a continuous need to understand how they can be exploited at runtime. As such, whilst research into inference time security is not, at a high level, a gap in this field time and funding resources should continue to be channeled into this space as AI systems continue to develop and new threats in this space emerge.

End of Lifecycle and Disposal Risks: This theme refers to the fact that, as AI systems may contain a significant amount of sensitive data, if these systems are not carefully handled at the end of their lifecycle it may become possible for an attacker to ‘scavenge’ these systems. This could allow both extraction of PII, or potentially reconstruction of the model itself. Whilst techniques such as machine unlearning have been studied for ML systems, these are far harder to apply to frontier AI such as LLMs, and so understanding how to safely and securely dispose of retiring AI systems is crucial for practitioners.

There are 76 papers well-aligned to this theme, and a significant proportion of papers concern the secure forgetting of data or model components. Examples include those explicitly connected to machine unlearning (13%), and several further papers considering topics linked to data forgetting (21%). More generally, end-of-lifecycle data security is a key theme in this topic area (11% of papers). There are gaps in this area, however, for example papers considering data management for model end of life (2%), and no keywords explicitly mentioned model disposal despite this being an emerging component in standards such as ETSI EN 304 223.

System Infrastructure Security: This theme refers to the attack surface of the system infrastructure underpinning a deployed AI model. Whilst the components of these systems have traditional cybersecurity vulnerabilities, for example misconfigured Cloud environments or insecure APIs, the integration of the AI model into these systems can increase the cybersecurity risk as it adds complexity to the attack surface due to the data pipelines and deployment environments they require. Due to the privileges they require to complete their tasks, this problem is likely to become worse with the growth of frontier systems such as Agentic-AI.

There are 132 well-aligned papers in this theme, which has already been identified as a gap at a thematic level. Threat (14%) or vulnerability (11%) modelling forms a key part of the literature in this area. Related to this, there is an explicit focus on a particular type of traditional cybersecurity attack on an AI system (15%), for instance Denial-of-Service; or defence mechanisms such as intrusion detection against this (14%). However, the majority of papers in this area relate to non-generative- or agentic-AI (89%). More work is potentially needed to understand how to map, and protect, the entire attack surfaces of Generative or Agentic-AI systems.

Security Risks Arising from Failure to Track Assets: This theme refers to two different components. The first element is the monitoring of AI systems over their lifecycle, for example to detect security issues arising from model or concept drift. The second is linked to the challenges an organisation faces when trying to inventory their models. Failing to do so gives rise to ‘Shadow AI’, and the use of undocumented models. These untracked assets lie outside the scope of security teams who are unaware of their presence, which may mean they are unpatched or lack threat monitoring, and thus represent a blind spot that attackers can target.

Only 24 papers are well aligned to this area, and of these papers there are only limited themes available. That said, monitoring and anomaly detection are well covered keywords (20% of papers). Similarly, security risks arising from drift in the model, or environment, over time, are well studied (16%). Monitoring performance over time is just one type of asset tracking, however. The other emphasis in this theme area is concerned with inventorying AI systems and assets to secure them. It appears that significantly less attention has been given to this area, with fewer papers considering inventorying models (7%). Understanding the security implications of undocumented AI usage across organisations is an important area that needs consideration, both from the technical and socio-technical cyber security research community.

Autonomous Agent Security: This theme refers to the security of AI systems that have an agent as a component, and where this agent can act autonomously, particularly emerging systems built upon Agentic-AI. In this paradigm the security risks grow significantly due to the creation of new attack vectors - for example tool or memory poisoning, indirect prompt injection, or the expansion of the system attack surface due to new concepts such as agent-to-agent operations. Security risks also arise from the nature of the systems themselves: the models underpinning these systems are probabilistic, and there is the possibility for rogue behaviour despite no adversary attacking the system.

Of the 91 papers in this theme, many consider adversarial attacks on some form on agent-based AI systems (20%). Few papers make use of keywords they are closely tied to emerging agentic systems (those based on generative AI systems such as LLMs, equipped with tools), for example tool (3%), or memory (2%), security. This shows how, to date, there has been only limited publication attention given to the security of this emerging form of AI  although this is entirely to be expected given agentic-AI is an emerging technology.

Security Risks Arising from End User: This theme refers to security challenges arising due to the way the end-user interacts with an AI system. For example, through an over-reliance on AI systems, known as automation bias, consumers and organisations are at risk of inadvertently trusting malicious or flawed outputs. This means that the end user themselves can become a security vulnerability when they interact with the AI systems, and this trust in these systems is something that attackers are actively seeking to exploit. An example consequence could be executing harmful LLM-generated code that interferes with system performance or leads to a negative outcome such as a data breach.

91 papers in the sample are well-aligned to this theme, and several of them (18%) are connected to the interaction between the end-user and exploits of vulnerabilities in the system. Two other key sub-themes in this area connect to end-user trust (7%) and decision making (6% of papers). On the other hand, there is only limited attention given to how the end-user themselves can pose a threat. For example, few papers explicitly focus on the security risks of running malicious AI generated code (1%), or the security implications of an end-user acting on hallucinated outputs (2%), and yet this is likely to become more common over the next 5 years, particularly as the use of ‘vibe-coding’ grows. Similarly, only a limited number papers are concerned with insider threats (2%). Thus, whilst this theme is already arguably under-represented in the literature in general, a finding of this review is that further research is needed to understand how the end-user can bring about security risks in AI systems through their actions, especially when their decision-making is based upon AI system outputs.

Model and System Design Security: This theme refers to the security of an AI system at its initial design stage. This requires, for example, following secure-by-design system principles, carefully considering details such as access control, or modelling and mitigating threats at the design stage. In doing so, designers can ensure that the architecture of the system itself is secure.

There were 55 papers returned that were well-aligned in this theme. Of these 55, many of these (50%) focus on the use of design and protocols for securing AI systems, and so secure-by-design and related notions is a well covered theme. The other area that has received significant attention is the security by design of non-frontier AI systems, i.e. those that are not LLMs or Agents (85% of papers). On the other hand, this could be framed as a gap as only 15% of papers cover these frontier systems. As shown by Figure 8, however, the number of papers published on Agentic security has more than tripled annually, and so this gap is closing.

Two other areas where there can be said to be less coverage are designs incorporating formal methods (4%) and how human factors (4%) of papers fit into secure design. Considering this latter sub-theme in tandem with the fact that security risks arising from end-users is a theme that has received comparatively less attention, means that this is a gap in this theme where more research would be advantageous.

Security Governance and Regulation: This theme refers to how new security regulations and governance frameworks are being developed to ensure the secure operation of AI. As these systems become widely deployed it is important to assess how well-covered they are through regulation and governance support, and also to highlight that organisations and nations that fail to adequately govern or regulate the use of AI systems may be exposing themselves to greater risk of attack.

As identified in the previous section, coverage of this theme is sparse, and there are only 21 well-aligned papers. At a high level, many of these papers consider some form of guidance on security and privacy for AI systems (48%). Given the sample size, it is difficult to identify gaps within this subtheme. It should be noted that a prominent area concerns trust and explainability (38% of papers). An area that has received less attention, on the other hand, concerns data security (9%), or related topics such as data sovereignty (5%), or privacy (9%).

One further area where limited attention has so far been received is cataloguing and making publicly-available known vulnerabilities in AI models. Whilst some early adopters such as the AVID and LM security databases are attempting to close this loop, at the time of writing, the number of vulnerabilities recorded by both bodies is low, and usage of these databases appears limited. As such, the adequate documenting and open-source publication of AI system vulnerabilities is an opportunity that should receive greater attention, in a similar vein to how the OECD’s AIM system is emerging as a database tracking AI security incidents.

5.1 Quantitative Findings

AI Security has garnered increasing levels of attention over the past five years, with over 3,000 papers published in 2024 and 2025, whereas in 2021 and 2022 numbers lagged at around 500 a year (Figure 3). Much of the small drop-off in publication volume in 2025 is down to conference publications. This section sets out notable statistics identified from the thematic analysis.

Figure 3: Search Results by Year

The papers returned in the search were mapped to their best aligned theme using semantic matching. This is shown in Figure 4. There are three levels of theme coverage. The top level, featuring Training Time Security, and Data/Privacy Risks, has received significantly more attention than other themes in this review with 41% and 26% respectively. The next layer down contains three themes that have received moderate levels of substantial attention. This encompasses Supply Chain Vulnerabilities, Alignment, and Inference Time Security. Each of these themes covers approximately 8% of the papers in this review. Finally, the third level of coverage contains the remaining seven themes, with these topics given comparatively less attention than the others in the study, and with increasingly diminishing returns. Coverage ranges from 2% for the Security Risks When Failing To Track Assets, to 0.5% for Security Governance and Regulation.

Figure 4: Paper Counts by Theme Alignment

The intention of this report is to understand the research landscape across a range of sectors, and the papers in this report constitute ones written by, and funded by, a range of institutions from academic, to corporations, to official institutions. Figure 5 plots number of papers each year according to their author affiliations. See that papers written exclusively by authors at academic institutions tend to dominate yearly publication volumes. Those papers that have authors at a mixture of institutions are also voluminous. Papers with at least one author affiliation to a corporation hit 900 or more in 2024 and over 500 in 2025, as well as those from a mixture of institutions excluding the corporate category. The data for this chart was prepared by an LLM - ChatGPT-5.4-mini - assigning a label to more than 15,000 individual affiliations in the dataset. These labels were one of: academic corporate/industry, government/public sector, healthcare, non-profit. A very small number (27) of classifications were labelled unknown. Because of low numbers, the healthcare, non-profit and unknown classes were grouped into ‘Other’ in the below plot.

Figure 5: Affiliation of Author(s) Per-Paper

Figure 6 shows the distribution of top funding bodies over papers. This represents the number of sources where an individual funding institution was noted as a funder at least once. Multiple funders per paper are possible. Not all papers in the dataset had funding information, which may be because the work was unfunded, or the authors did not disclose it. 1976 papers had no funding information available. Funding institutions in China and the United States have funded the most papers, with those in South Korea, Australia, Singapore and Europe also included in the top 15.   

Figure 6: Top 15 Funders by Number of Papers.

Figure 7, below, shows the geographic variability of the dataset, depicting the country of author’s affiliation by paper.^{[footnote 4]} A similar picture to funders is visible, with China and the United States dominating, with Australia, Singapore, and South Korea also making a strong showing. The United Kingdom and Hong Kong round out the top 7.

Figure 7: Country of Affiliation of Author(s) Per-Paper

Examining the results by AI system (assigned by an LLM - ChatGPT-5.4-mini - based on the title, abstract and keywords, if available, of each paper), as depicted in Figure  4,it is seen that Deep Learning and Machine Learning are by far the best represented. Generative AI and Computer Vision are associated with between 1,000 and 2,000 papers. Agentic-AI is the least well represented, with just 122 papers about this fast-growing subject.

Figure 8: Search Results by AI System

6. Gap Analysis

This report has covered a substantial volume of literature relating to the cyber security of AI, covering a broad range of themes capturing security across the lifecycle of these systems. At a high level, the review has highlighted several themes where further attention is required, for example, the security of system infrastructure or mitigating risks arising from end-use or at the end of the system’s lifecycle.  In addition to the five major gaps in the literature outlined below, the report has also set out a gap analysis on areas that are not covered in detail from the research themes identified from this study.

1. Data Integrity: Research is needed to develop methods to formally guarantee, or verify, the integrity of data used by AI systems. This includes both the data used to train the model, or preserving the security of the supply chain by verifying the components of the system (such as the model) are secure.

2. AI System Infrastructure Security: Whilst there has been some research exploring how to ensure the cybersecurity of infrastructure underpinning deployed machine or deep learning models, comparatively less attention has been given to emerging generative or Agentic-AI systems. As such, there is a need for further research in this area. This includes both runtime security and managing the attack surfaces of these systems, and developing a better understanding of how frontier AI such as LLMs change what is required for a system to be secure-by-design.

3. Securing Against End-User Risks: More consideration needs to be given to how the end-user can themselves pose a security risk to AI systems. As discussed in the gap analysis there has been limited attention given to concerns such as end-users relying upon hallucinated or malicious outputs. The probability of these issues arising is directly tied to other aspects of AI security, such as supply chain risks or using models trained on poisoned data. Although not strictly a system-level security concern, this topic also ties to one aspect of another gap, shadow AI, and the potential for end-user to be accessing undocumented AI models from within their organisation, creating further security and privacy risks.

4. Data and Model Provenance: As the models used within AI systems are increasingly being sourced externally, for example from Hugging Face or another third-party provider, there is an increased need to understand the provenance of these systems and its security implications. There are also opportunities for standardisation and regulation in this area, and for the methods to automate the tracking and detection of models that pose security threats, in a similar vein to malware.

5. Model Disposal: Whilst some attention has been given to machine unlearning for AI systems such as machine and deep learners, considerably less attention has been given to model disposal for frontier-AI systems. As such, research is needed to understand model disposal, and its security implications, for these systems.

Overall, these recommendations present opportunities for researchers across a range of disciplines tied to cybersecurity. For example, they vary from technical mathematical methods to guarantee training data or model integrity, and security for system infrastructure; to answering the socio-technical questions needed to ensure systems are protect against end-user risks. These recommendations therefore provide a fantastic platform from which to engage the research community in order to enhance the cybersecurity of AI.

The review also found significant gaps on the cyber security of Agentic-AI systems, and the new attack surfaces and risks that these systems create. Examples of this include the security of the agent themselves, the tools they use to interact with their environment, and inter-agent communication and operation. The reason that this area has received less coverage, however, is that this technology is still emerging. Despite this, agentic systems are expected to become a widely adopted AI system over the next 2-3 years, and so the authors anticipate that research into their cybersecurity will increase in pace as deployment increases. This is supported by the findings of this review, which show a 216% increase in the number of papers published on agentic-AI security in 2025 compared to 2024; this is the largest year-on-year increase across the 12 themes of this review.

In addition to these research recommendations, this review also highlighted a non-research area that would benefit from policy direction: establishing a widely used database of AI system vulnerabilities, akin to the way that CVEs are tracked for other systems. Whilst the review has highlighted example databases, usage of these platforms appears limited. The motivation behind this recommendation is to move away from anecdotes about AI security to a formal model that captures known vulnerabilities in/to these systems, in a similar fashion to how the OECD’s AIM platform is recording instances of incidents involving AI. It cuts across all of the themes presented in this report, and could be used by decision makers deploying AI, and those designing or using these systems, to enhance their security.

7. Gap analysis for least covered research themes

The five themes that have received comparatively less coverage could also be an opportunity for further research. Chief among these is System Infrastructure Security. This theme captures how ‘traditional’ cyber security vulnerabilities manifest as risks to AI systems, for example insecure container orchestration, or insecure APIs. This is a critical research question as society is expected to deploy, and rely upon, AI systems for daily life. As such, this review finds this to be a critical area where further efforts are required, particularly for emerging frontier-AI systems.

Regarding the Governance and Regulation theme, although there are relatively few sources covering this area, it is not necessarily an issue. This is because this is an area where a small number of valuable frameworks and regulations would be sufficient to set the standards in this area. As such, it can be argued that frequency of publication here is not correlated with how well the theme is covered from the perspective of knowledge. A similar argument can be made for the Model/System Design Security theme. This is because much of this knowledge is likely to be proprietary and held by organisations rather than published in the public domain. Again, this means that a low publication count in this area does not necessarily correspond to a gap in knowledge.

There are 3 additional themes that present a potential area for future work: Security Risks Arising from End-Use, and End of Lifecycle and Model Disposal Risks, both which are particularly relevant as AI systems are deployed across society; and System Infrastructure Security. Finally, it should be noted that no argument is being advanced to suggest that attention on critical themes such as Training or Inference Time Security should be reduced. These represent a significant attack surface for AI systems and are quite rightly given attention.

8. References

In addition to the 9109 papers in the search, the following sources have been used for this report:

Bitton, Ron, Nadav Maman, Inderjeet Singh, Satoru Momiyama, Yuval Elovici, and Asaf Shabtai. 2023. “Evaluating the Cybersecurity Risk of Real-World, Machine Learning Production Systems.” ACM Computing Surveys 55 (9): 1–36.

Camacho, Jose Manuel, Aitor Couce-Vieira, David Arroyo, and David Rios Insua. 2026. “A Cybersecurity Risk Analysis Framework for Systems with Artificial Intelligence Components.” International Transactions in Operational Research 33 (2): 798–825.

Centre, National Cyber Security. 2026. Artificial Intelligence. NCSC. https://www.ncsc.gov.uk/section/advice-guidance/all-topics?topics=Artificial%20intelligence&sort=date%2Bdesc.

Deng, Zehang, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. 2025. “Ai Agents Under Threat: A Survey of Key Security Challenges and Future Pathways.” ACM Computing Surveys 57 (7): 1–36.

ETSI. 2025. Securing Artificial Intelligence (SAI); Baseline Cyber Security Requirements for AI Models and Systems Artificial Intelligence. ETSI. https://www.etsi.org/deliver/etsi_ts/104200_104299/104223/01.01.01_60/ts_104223v010101p.pdf.

Google. 2025. Google’s Secure AI Framework (SAIF). Google. https://safety.google/intl/en-GB_ALL/safety/saif/.

Government, UK. 2025. AI Playbook for the UK Government. UK Government. https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government/artificial-intelligence-playbook-for-the-uk-government-html.

ISO/IEC. 2022. ISO/IEC 22989: Artificial Intelligence. ISO/IEC. https://www.iso.org/standard/74296.html.

———. 2023. ISO/IEC 42001: AI Management. ISO/IEC. https://www.iso.org/standard/42001.

Ji, Jiaming, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, et al. 2025. “AI Alignment: A Comprehensive Survey.” https://arxiv.org/abs/2310.19852.

Kayode, B, NT Adebola, and S Akerele. 2025. “The State of AI-Driven Cybersecurity: Trends, Challenges, and Opportunities.” J Artif Intell Mach Learn & Data Sci 3 (2): 2731–39.

Martinez del Rincon, Jesus, Ehsan Nowroozi, Eleni Kamenou, Ihsen Alouani, Sandeep Gupta, and Paul Miller. 2024. Study of Research and Guidance on the Cyber Security of AI. Centre for Secure Information Technologies, Queens University, Belfast. https://www.gov.uk/government/publications/research-on-the-cyber-security-of-ai.

MITRE. 2026. MITRE ATLAS. MITRE. https://atlas.mitre.org/.

Narula, Sidhant, Mohammad Ghasemigol, Javier Carnerero-Cano, Amanda Minnich, Emil Lupu, and Daniel Takabi. 2025. “Exploring AI Security: A Systematic Mapping Study.” IEEE Access.

NIST. 2023. NIST AI Risk Management Framework. NIST. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf.

OWASP. 2026. OWASP Top 10 for Agentic Applications. OWASP. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/.

R Core Team. 2019. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org.

Reimers, Nils, and Iryna Gurevych. 2019. “Sentence-Bert: Sentence Embeddings Using Siamese Bert-Networks.” In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3982–92.

Song, Yangqiu, Shyam Upadhyay, Haoruo Peng, Stephen Mayhew, and Dan Roth. 2019. “Toward Any-Language Zero-Shot Topic Classification of Textual Documents.” Artificial Intelligence 274: 133–50.

Stuart Russell, Marko Grobelnik, Karine Perset. 2023. Updates to the OECD’s Definition of an AI System Explained. OECD. https://oecd.ai/en/wonk/ai-system-definition-update.

Wickham, Hadley, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain François, Garrett Grolemund, et al. 2019. “Welcome to the Tidyverse.” Journal of Open Source Software 4 (43): 1686.

9. Appendix A: Literature search

Web of Science search terms:

(SO=“Artificial Intelligence” OR SO=“ACM Computing Surveys” OR SO=“ACM Comput Surv” OR SO=“Computers & Security” OR SO=“Data Mining and Knowledge Discovery” OR SO=“Data Min Knowl Discov” OR SO=“Digital Investigation” OR SO=“Foundations and Trends in Machine Learning” OR SO=“Found Trends Mach Learn” OR SO=“IEEE Communications Surveys and Tutorials” OR SO=“IEEE Commun Surv Tutor” OR SO=“IEEE Transactions on Dependable and Secure Computing” OR SO=“IEEE Trans Dependable Secure Comput” OR SO=“IEEE Transactions on Information Forensics and Security” OR SO=“IEEE Trans Inf Forensics Secur” OR SO=“IEEE Transactions on Neural Networks and Learning Systems” OR SO=“IEEE Trans Neural Netw Learn Syst” OR SO=“IEEE Transactions on Pattern Analysis and Machine Intelligence” OR SO=“IEEE Trans Pattern Anal Mach Intell” OR SO=“IET Information Security” OR SO=“IET Inf Secur” OR SO=“Information Fusion” OR SO=“International Journal of Information Security” OR SO=“Int J Inf Secur” OR SO=“Journal of Computer Security” OR SO=“J Comput Secur” OR SO=“Journal of Cryptographic Engineering” OR SO=“J Cryptogr Eng” OR SO=“Journal of Cybersecurity” OR SO=“Journal of Information Security and Applications” OR SO=“J Inf Secur Appl” OR SO=“Journal of Machine Learning Research” OR SO=“J Mach Learn Res” OR SO=“Knowledge-Based Systems” OR SO=“Knowl Based Syst” OR SO=“Machine Learning” OR SO=“Nature Machine Intelligence” OR SO=“Nat Mach Intell” OR SO=“Neural Networks” OR SO=“Pattern Recognition” OR SO=“Security and Communication Networks” OR SO=“Secur Commun Netw” OR SO=“AAAI Conference on Artificial Intelligence” OR SO=“AAAI” OR SO=“ACM Conference on Computer and Communications Security” OR SO=“ACM CCS” OR SO=“ACM SIGKDD Conference on Knowledge Discovery and Data Mining” OR SO=“KDD” OR SO=“Annual Computer Security Applications Conference” OR SO=“ACSAC” OR SO=“Annual Meeting of the Association for Computational Linguistics” OR SO=“ACL” OR SO=“Conference on Empirical Methods in Natural Language Processing” OR SO=“EMNLP” OR SO=“Conference on Uncertainty in Artificial Intelligence” OR SO=“UAI” OR SO=“European Conference on Computer Vision” OR SO=“ECCV” OR SO=“European Cryptology Conference” OR SO=“EUROCRYPT” OR SO=“IEEE Conference on Communications and Network Security” OR SO=“IEEE CNS” OR SO=“IEEE Conference on Computer Vision and Pattern Recognition” OR SO=“CVPR” OR SO=“IEEE Symposium on Security and Privacy” OR SO=“IEEE S&P” OR SO=“International Conference on Artificial Intelligence and Statistics” OR SO=“AISTATS” OR SO=“International Conference on Computer Vision” OR SO=“ICCV” OR SO=“International Conference on Learning Representations” OR SO=“ICLR” OR SO=“International Conference on Machine Learning” OR SO=“ICML” OR SO=“International Conference on Security and Privacy in Communication Networks” OR SO=“SecureComm” OR SO=“International Conference on the Theory and Application of Cryptology and Information Security” OR SO=“ASIACRYPT” OR SO=“International Cryptology Conference” OR SO=“CRYPTO” OR SO=“International Joint Conference on Artificial Intelligence” OR SO=“IJCAI” OR SO=“International Symposium on Research in Attacks, Intrusions and Defenses” OR SO=“RAID” OR SO=“Network and Distributed System Security Symposium” OR SO=“NDSS” OR SO=“Neural Information Processing Systems” OR SO=“NeurIPS” OR SO=“Privacy Enhancing Technologies Symposium” OR SO=“PETS” OR SO=“USENIX Security Symposium” OR SO=“USENIX Security”) AND ( TI=( “AI security” OR “artificial intelligence security” OR “adversarial machine learning” OR “AI robustness” OR “artificial intelligence robustness” OR “model hallucinations” OR “unaligned AI” OR “unaligned artificial intelligence” OR “data poisoning” OR “training data integrity” OR “backdoor attacks” OR “trojaned models” OR “trojan attacks” OR “model inversion” OR “model extraction” OR “model theft” OR “inference attacks” OR “membership inference” OR “sponge attacks” OR “AI data leakage” OR “artificial intelligence data leakage” OR “PII exposure” OR “personally identifiable information exposure” OR “training data privacy” OR “reconstruction attacks” OR “AI API security” OR “artificial intelligence application programming interfaces security” OR “GPU security” OR “graphics processing unit security” OR “MLOps vulnerabilities” OR “machine learning operations vulnerabilities” OR “AI supply chain” OR “artificial intelligence supply chain” OR “third-party model risk” OR “model decommissioning” OR “machine unlearning” OR “algorithmic disgorgement” OR “secure model deletion” OR “shadow AI” OR “shadow artificial intelligence” OR “model drift vulnerabilities” OR “performance degradation” OR “threat model” OR “training attacks” OR “causative attacks” OR “poisoning attacks” OR “testing attacks” OR “exploratory attacks” OR “white-box attacks” OR “white box attacks” OR “black-box attacks” OR “black box attacks” OR “oracle attacks” OR ” training data integrity” OR “label flipping” OR “unauthorised training data” OR “model skewing” OR “model update poisoning” OR “PII leakage” OR “personally identifiable information leakage” OR “model provenance” OR “huggingface model risks” OR “hugging face model risks” OR “model deletion” OR “model disposal” OR “training data disposal” OR “AI alignment”
 OR “artificial intelligence alignment” OR (alignment AND (security OR risk OR vulnerability) AND (AI OR “artificial intelligence”))

) OR AB=( “AI security” OR “artificial intelligence security” OR “adversarial machine learning” OR “AI robustness” OR “artificial intelligence robustness” OR “model hallucinations” OR “unaligned AI” OR “unaligned artificial intelligence” OR “data poisoning” OR “training data integrity” OR “backdoor attacks” OR “trojaned models” OR “trojan attacks” OR “model inversion” OR “model extraction” OR “model theft” OR “inference attacks” OR “membership inference” OR “sponge attacks” OR “AI data leakage” OR “artificial intelligence data leakage” OR “PII exposure” OR “personally identifiable information exposure” OR “training data privacy” OR “reconstruction attacks” OR “AI API security” OR “artificial intelligence application programming interfaces security” OR “GPU security” OR “graphics processing unit security” OR “MLOps vulnerabilities” OR “machine learning operations vulnerabilities” OR “AI supply chain” OR “artificial intelligence supply chain” OR “third-party model risk” OR “model decommissioning” OR “machine unlearning” OR “algorithmic disgorgement” OR “secure model deletion” OR “shadow AI” OR “shadow artificial intelligence” OR “model drift vulnerabilities” OR “performance degradation” OR “threat model” OR “training attacks” OR “causative attacks” OR “poisoning attacks” OR “testing attacks” OR “exploratory attacks” OR “white-box attacks” OR “white box attacks” OR “black-box attacks” OR “black box attacks” OR “oracle attacks” OR ” training data integrity” OR “label flipping” OR “unauthorised training data” OR “model skewing” OR “model update poisoning” OR “PII leakage” OR “personally identifiable information leakage” OR “model provenance” OR “huggingface model risks” OR “hugging face model risks” OR “model deletion” OR “model disposal” OR “training data disposal” OR “AI alignment”
 OR “artificial intelligence alignment” OR (alignment AND (security OR risk OR vulnerability) AND (AI OR “artificial intelligence”)) ) OR AK=( “AI security” OR “artificial intelligence security” OR “adversarial machine learning” OR “AI robustness” OR “artificial intelligence robustness” OR “model hallucinations” OR “unaligned AI” OR “unaligned artificial intelligence” OR “data poisoning” OR “training data integrity” OR “backdoor attacks” OR “trojaned models” OR “trojan attacks” OR “model inversion” OR “model extraction” OR “model theft” OR “inference attacks” OR “membership inference” OR “sponge attacks” OR “AI data leakage” OR “artificial intelligence data leakage” OR “PII exposure” OR “personally identifiable information exposure” OR “training data privacy” OR “reconstruction attacks” OR “AI API security” OR “artificial intelligence application programming interfaces security” OR “GPU security” OR “graphics processing unit security” OR “MLOps vulnerabilities” OR “machine learning operations vulnerabilities” OR “AI supply chain” OR “artificial intelligence supply chain” OR “third-party model risk” OR “model decommissioning” OR “machine unlearning” OR “algorithmic disgorgement” OR “secure model deletion” OR “shadow AI” OR “shadow artificial intelligence” OR “model drift vulnerabilities” OR “performance degradation” OR “threat model” OR “training attacks” OR “causative attacks” OR “poisoning attacks” OR “testing attacks” OR “exploratory attacks” OR “white-box attacks” OR “white box attacks” OR “black-box attacks” OR “black box attacks” OR “oracle attacks” OR ” training data integrity” OR “label flipping” OR “unauthorised training data” OR “model skewing” OR “model update poisoning” OR “PII leakage” OR “personally identifiable information leakage” OR “model provenance” OR “huggingface model risks” OR “hugging face model risks” OR “model deletion” OR “model disposal” OR “training data disposal” OR “AI alignment”
 OR “artificial intelligence alignment” OR (alignment AND (security OR risk OR vulnerability) AND (AI OR “artificial intelligence”)) ) )

Scopus search terms

( ( ( EXACTSRCTITLE(“Artificial Intelligence”) OR EXACTSRCTITLE(“ACM Computing Surveys”) OR EXACTSRCTITLE(“ACM Comput Surv”) OR EXACTSRCTITLE(“Computers & Security”) OR EXACTSRCTITLE(“Computers and Security”) OR EXACTSRCTITLE(“Data Mining and Knowledge Discovery”) OR EXACTSRCTITLE(“Data Min Knowl Discov”) OR EXACTSRCTITLE(“Digital Investigation”) OR EXACTSRCTITLE(“Foundations and Trends in Machine Learning”) OR EXACTSRCTITLE(“Found Trends Mach Learn”) OR EXACTSRCTITLE(“IEEE Communications Surveys and Tutorials”) OR EXACTSRCTITLE(“IEEE Commun Surv Tutor”) OR EXACTSRCTITLE(“IEEE Transactions on Dependable and Secure Computing”) OR EXACTSRCTITLE(“IEEE Trans Dependable Secure Comput”) OR EXACTSRCTITLE(“IEEE Transactions on Information Forensics and Security”) OR EXACTSRCTITLE(“IEEE Trans Inf Forensics Secur”) OR EXACTSRCTITLE(“IEEE Transactions on Neural Networks and Learning Systems”) OR EXACTSRCTITLE(“IEEE Trans Neural Netw Learn Syst”) OR EXACTSRCTITLE(“IEEE Transactions on Pattern Analysis and Machine Intelligence”) OR EXACTSRCTITLE(“IEEE Trans Pattern Anal Mach Intell”) OR EXACTSRCTITLE(“IET Information Security”) OR EXACTSRCTITLE(“IET Inf Secur”) OR EXACTSRCTITLE(“Information Fusion”) OR EXACTSRCTITLE(“International Journal of Information Security”) OR EXACTSRCTITLE(“Int J Inf Secur”) OR EXACTSRCTITLE(“Journal of Computer Security”) OR EXACTSRCTITLE(“J Comput Secur”) OR EXACTSRCTITLE(“Journal of Cryptographic Engineering”) OR EXACTSRCTITLE(“J Cryptogr Eng”) OR EXACTSRCTITLE(“Journal of Cybersecurity”) OR EXACTSRCTITLE(“Journal of Information Security and Applications”) OR EXACTSRCTITLE(“J Inf Secur Appl”) OR EXACTSRCTITLE(“Journal of Machine Learning Research”) OR EXACTSRCTITLE(“J Mach Learn Res”) OR EXACTSRCTITLE(“Knowledge-Based Systems”) OR EXACTSRCTITLE(“Knowl Based Syst”) OR EXACTSRCTITLE(“Machine Learning”) OR EXACTSRCTITLE(“Nature Machine Intelligence”) OR EXACTSRCTITLE(“Nat Mach Intell”) OR EXACTSRCTITLE(“Neural Networks”) OR EXACTSRCTITLE(“Pattern Recognition”) OR EXACTSRCTITLE(“Security and Communication Networks”) OR EXACTSRCTITLE(“Secur Commun Netw”) OR EXACTSRCTITLE(“AAAI Conference on Artificial Intelligence”) OR EXACTSRCTITLE(“AAAI”) OR EXACTSRCTITLE(“ACM Conference on Computer and Communications Security”) OR EXACTSRCTITLE(“ACM CCS”) OR EXACTSRCTITLE(“ACM SIGKDD Conference on Knowledge Discovery and Data Mining”) OR EXACTSRCTITLE(“KDD”) OR EXACTSRCTITLE(“Annual Computer Security Applications Conference”) OR EXACTSRCTITLE(“ACSAC”) OR EXACTSRCTITLE(“Annual Meeting of the Association for Computational Linguistics”) OR EXACTSRCTITLE(“ACL”) OR EXACTSRCTITLE(“Conference on Empirical Methods in Natural Language Processing”) OR EXACTSRCTITLE(“EMNLP”) OR EXACTSRCTITLE(“Conference on Uncertainty in Artificial Intelligence”) OR EXACTSRCTITLE(“UAI”) OR EXACTSRCTITLE(“European Conference on Computer Vision”) OR EXACTSRCTITLE(“ECCV”) OR EXACTSRCTITLE(“European Cryptology Conference”) OR EXACTSRCTITLE(“EUROCRYPT”) OR EXACTSRCTITLE(“IEEE Conference on Communications and Network Security”) OR EXACTSRCTITLE(“IEEE CNS”) OR EXACTSRCTITLE(“IEEE Conference on Computer Vision and Pattern Recognition”) OR EXACTSRCTITLE(“CVPR”) OR EXACTSRCTITLE(“IEEE Symposium on Security and Privacy”) OR EXACTSRCTITLE(“IEEE S&P”) OR EXACTSRCTITLE(“International Conference on Artificial Intelligence and Statistics”) OR EXACTSRCTITLE(“AISTATS”) OR EXACTSRCTITLE(“International Conference on Computer Vision”) OR EXACTSRCTITLE(“ICCV”) OR EXACTSRCTITLE(“International Conference on Learning Representations”) OR EXACTSRCTITLE(“ICLR”) OR EXACTSRCTITLE(“International Conference on Machine Learning”) OR EXACTSRCTITLE(“ICML”) OR EXACTSRCTITLE(“International Conference on Security and Privacy in Communication Networks”) OR EXACTSRCTITLE(“SecureComm”) OR EXACTSRCTITLE(“International Conference on the Theory and Application of Cryptology and Information Security”) OR EXACTSRCTITLE(“ASIACRYPT”) OR EXACTSRCTITLE(“International Cryptology Conference”) OR EXACTSRCTITLE(“CRYPTO”) OR EXACTSRCTITLE(“International Joint Conference on Artificial Intelligence”) OR EXACTSRCTITLE(“IJCAI”) OR EXACTSRCTITLE(“International Symposium on Research in Attacks, Intrusions and Defenses”) OR EXACTSRCTITLE(“RAID”) OR EXACTSRCTITLE(“Network and Distributed System Security Symposium”) OR EXACTSRCTITLE(“NDSS”) OR EXACTSRCTITLE(“Neural Information Processing Systems”) OR EXACTSRCTITLE(“NeurIPS”) OR EXACTSRCTITLE(“Privacy Enhancing Technologies Symposium”) OR EXACTSRCTITLE(“PETS”) OR EXACTSRCTITLE(“Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining”) OR EXACTSRCTITLE(“USENIX Security Symposium”) OR EXACTSRCTITLE(“USENIX Security”) ) AND (SRCTYPE(j) OR SRCTYPE(p)) ) OR ( ( CONFNAME(“International Conference on Security and Privacy in Communication Networks”) OR CONFNAME(“SecureComm”) OR CONFNAME(“European Conference on Computer Vision”) OR CONFNAME(“ECCV”) OR CONFNAME(“International Cryptology Conference”) OR CONFNAME(“CRYPTO”) OR CONFNAME(“European Cryptology Conference”) OR CONFNAME(“EUROCRYPT”) OR CONFNAME(“International Conference on the Theory and Application of Cryptology”) OR CONFNAME(“ASIACRYPT”) OR CONFNAME(“Privacy Enhancing Technologies Symposium”) OR CONFNAME(“PETS”) OR CONFNAME(“RAID”) OR CONFNAME(“International Symposium on Research in Attacks, Intrusions and Defenses”) ) AND (DOCTYPE(ar) OR DOCTYPE(cp)) ) AND (TITLE-ABS-KEY( “AI security” OR “artificial intelligence security” OR “adversarial machine learning” OR “AI robustness” OR “artificial intelligence robustness” OR “model hallucinations” OR “unaligned AI” OR “unaligned artificial intelligence” OR “data poisoning” OR “training data integrity” OR “backdoor attacks” OR “trojaned models” OR “trojan attacks” OR “model inversion” OR “model extraction” OR “model theft” OR “inference attacks” OR “membership inference” OR “sponge attacks” OR “AI data leakage” OR “artificial intelligence data leakage” OR “PII exposure” OR “personally identifiable information exposure” OR “training data privacy” OR “reconstruction attacks” OR “AI API security” OR “artificial intelligence application programming interfaces security” OR “GPU security” OR “graphics processing unit security” OR “MLOps vulnerabilities” OR “machine learning operations vulnerabilities” OR “AI supply chain” OR “artificial intelligence supply chain” OR “third-party model risk” OR “model decommissioning” OR “machine unlearning” OR “algorithmic disgorgement” OR “secure model deletion” OR “shadow AI” OR “shadow artificial intelligence” OR “model drift vulnerabilities” OR “performance degradation” OR “threat model” OR “training attacks” OR “causative attacks” OR “poisoning attacks” OR “testing attacks” OR “exploratory attacks” OR “white-box attacks” OR “white box attacks” OR “black-box attacks” OR “black box attacks” OR “oracle attacks” OR “training data integrity” OR “label flipping” OR “unauthorised training data” OR “model skewing” OR “model update poisoning” OR “PII leakage” OR “personally identifiable information leakage” OR “model provenance” OR “huggingface model risks” OR “hugging face model risks” OR “model deletion” OR “model disposal” OR “training data disposal” OR “AI alignment” OR “artificial intelligence alignment” OR (alignment AND (security OR risk OR vulnerability) AND (AI OR “artificial intelligence”)) )) )

---

1. In line with the OECD definition of an AI system (Stuart Russell 2023) and the ETSI standard on securing artificial intelligence the report defines an AI system as “An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.” ↩

2. A similar approach, that of categorising text by using short descriptions of categories and calculating semantic similarity (rather than, for example, a set of fully labelled training data) is explored in (Song et al. 2019). The exact descriptions for theme matching are available in the source code. ↩

3. The method employed semantic embedding via a sentence transformer model (Reimers and Gurevych 2019) in Python (via R). In this study, paper was mapped to the theme that it was most closely aligned with using an alignment score based on cosine similarity. The threshold for a paper to be deemed well aligned is a cosine similarity score of 0.3 or more. This was verified by manual inspections of random samples. The sentence transformer model used was the all-MiniLM-L6-v2. Details available at https://www.sbert.net/docs/sentence_transformer/pretrained_models.html (url last accessed March 9, 2026). ↩

4. As papers usually have multiple authors, multiple affiliations per paper are possible. However, the counts have been limited to a maximum of one for each country on a paper. For example, if authors from three UK institutions wrote a paper, that would contribute a count of one to the UK in the plot below. If authors from institutions in China, the United States and Australia were co-authors on a paper, then that would count one each for the respective countries. In the dataset, 99 papers were missing affiliations information and 651 affiliations were missing country affiliation. ↩