Exactly How Is AI Being Attacked? Lessons from MITRE ATLAS
In May 2025, researchers at Aim Security sent an ordinary-looking business email to someone using Microsoft 365 Copilot. The recipient never opened it, but Microsoft's AI assistant (Copilot) pulled the email into its working context while answering an unrelated question, followed the instructions surreptitiously hidden inside it by the researchers, searched everything that the recipient's user account could reach across Microsoft 365, and sent what it found to a web address the researchers controlled. In doing so, the data was exfiltrated past Microsoft's injection defenses, its link redaction, and its content security rules. Microsoft issued the flaw (called "EchoLeak") a vulnerability number and fixed it, and there's no evidence anyone used it in the wild. But the clear lesson is that AI has introduced novel ways to attack the systems that rely on it. In this case, the researchers showed that RAG (Retrieval-Augmented Generation) can be "tricked" into executing commands by an attacker.
Cases such as this one show how attacks on AI systems are possible, but it says nothing about how often such attacks happen, which parts of those systems are most exposed, or whether anyone has executed those attacks against real-life systems rather than in a lab. Risk professionals have to answer these questions. Two things will help them: an understanding of the attack surface (see last week’s blog post), and a catalog of the methods used against that attack surface. I’ll discuss the latter in this post, along with my general observations about what the data in that catalog means for risk professionals
Introducing MITRE ATLAS
MITRE ATLAS (Adversarial Threat Language for AI Systems) is a catalog of attack methods (tactics, techniques, and procedures, or TTPs) used against AI systems. ATLAS is similar to MITRE ATT&CK (Adversarial Tactics Techniques & Common Knowledge), which describes TTPs targeting enterprise, mobile, and industrial control systems. Like ATT&CK which came before it, ATLAS provides a growing catalog of TTPs, mitigations, and case studies to help risk professionals assess and mitigate threats to their systems, but scoped to AI ones.
To write this post, I used Claude to analyze the ATLAS catalog. I focused on how AI-enabled systems are being attacked and what parts of the AI attack surface appear most exposed. Do you need to worry more about your model providers being compromised, or your own AI-enabled systems that rely on those models? Where should you focus your risk mitigation efforts? Which attack vectors should you prioritize for analysis?
What MITRE ATLAS Measures
Before I talk about what my analysis revealed, let me share a brief overview of data from the ATLAS catalog. I won't give you a primer on ATLAS, but I'll share some details here to provide the necessary context.
The ATLAS release dated September 15, 2026 carries 208 technique entries, made up of 120 techniques and 88 subtechniques (the narrower variations beneath them), alongside 40 mitigations and 73 case studies. This growing catalog is extraordinarily helpful for understanding not only the overall variety of TTPs, but also which types (platforms) of AI systems are being attacked and the maturity of each TTP.
Let's look at these two TTP attributes, platforms and maturity, which you can use as filters on the ATLAS website (as shown below).

The platform tag describes the type of system where the attacker operates:
|
Platform |
Definition |
Count |
|---|---|---|
|
Predictive AI |
The attacker is working against the parts of the system that analyze data to classify, score, rank, forecast, or recommend an outcome. MITRE names the targets as the features, the model, the training data, the inference logic, and the decision thresholds. Fraud scoring and underwriting live here. |
139 |
|
Generative AI |
The parts that produce new content, whether text, code, images, audio, or video, along with the prompts, the model's output, its safety controls, the material it retrieves to answer with, and the pipeline that generates the result. |
106 |
|
Agentic AI |
The parts that plan, decide, and carry out multi-step actions on their own, usually with memory, tools, and some workflow orchestration behind them. The targets MITRE lists are the agent's logic, its planning loop, its tool integrations, and its control policies. |
85 |
|
Enterprise |
The ordinary organizational IT inside an AI-enabled system rather than the AI-specific pieces: user endpoints, servers, networks, identity systems, business applications, and cloud services. |
82 |
A technique can carry more than one tag, so they don't add to 208 in the table above. Interestingly, ATLAS began in 2021 as a list of attacks on machine learning models (predictive AI), but it now tags two-thirds of its entries to Generative AI and Agentic AI.
The second tag is a maturity rating, as defined below:
|
Maturity |
Definition |
Count |
|---|---|---|
|
Feasible |
Researchers have shown the attack can work. |
24 |
|
Demonstrated |
A researcher made it work against a realistic system. |
83 |
|
Realized |
An attacker used it in a real incident. |
101 |
The Model Is Rarely the Target
The 139 Agentic AI tags and 106 Generative AI tags above seem to suggest the model is what's most at risk of getting attacked, and that risk leaders should focus on the Model Development lifecycle from my previous blog post. In other words, the risk implied by ATLAS is mostly with those who develop and train AI models, which would make AI risk management a third-party risk management endeavor for organizations that buy their models from companies such as Anthropic, OpenAI, Google, or other companies. But that would be the wrong conclusion. A platform tag names the types of components the attacker is operating against, but the tags do not neatly correspond to either of the Model Development or Application Runtime lifecycles.
To understand this better, consider the specific targets that MITRE lists in the definitions for its Generative AI and Agentic AI tags:
- Prompts. Everything sent to the model on a given request, including your developers' system prompt.
- Model outputs. What the model returns, and everything that travels with it.
- Retrieval context. The documents a system looks up and hands to the model to use to come up with its response.
- Safety controls. The guardrails that evaluate prompts, retrieved context, outputs, and agent actions before any of them is accepted, executed, or shown. These guardrails sit between users, tools, and the model. They may be built into the model software or included in the software built around the model, or (often) both.
- Generation pipelines. The chain of services that turns a request into a finished response.
- Agent logic. The code and templates that assemble what the model actually sees. MITRE's nearest definition names "templates, role delimiters, embedded system instructions, tokenizer settings, tool-call formatting".
- Planning loops. The cycle in which an agent picks a next step, acts, reads the result, and picks the following step.
- Tool integrations. The built-in integrations, locally installed packages, remotely hosted services, Model Context Protocol servers and tools, and agent skills.
- Control policies. The rules fixing what an agent may do and how far its authority can stretch.
Most of these targets are or often include elements that are part of the Application Runtime lifecycle. For example, prompts include your application's system prompt that's written and maintained by your developers, not those who developed the model. Safety controls include any guardrails embedded in the model but also any guardrails built into your application that uses the model. And so on. So it would be wrong to conclude that a majority of the techniques in ATLAS must be mitigated by the model developer. Many can be, but most also require mitigation by the application developer (i.e., your organization, if it builds apps that rely on AI models).
It's even more instructive to classify the 208 techniques by the lifecycle from last week's post. I've included the lifecycle diagram below for easy reference.

ATLAS carries no field for my stages, so the classification is mine (Claude's actually, but reviewed by me). Each technique is assigned to the primary stage whose component it acts on. Ninety-three of the 208 techniques are not classified to any of the attack surface stages, and they split in two. Twenty act on the hosting layer, by which I mean the ordinary IT the system runs on: endpoints, servers, networks, identity systems and cloud services. MITRE's Enterprise platform tag treats that as part of an AI-enabled system, and it is, but it carries all eight stages rather than sitting inside one of them. The other 73 are attacker-side work, meaning reconnaissance, resource development, attack preparation, and delivery aimed at people rather than at a component of the system.
Of the 115 that do land on a stage, 76 are in the runtime half, meaning the prompt and context, the tools, the agent's identity, and the system that consumes the output. The other 39 are in the development half. Here is a table detailing the mapping of techniques to the attack surface stages:
|
Attack surface stage |
Feasible |
Demonstrated |
Realized |
Total |
|---|---|---|---|---|
|
1. Training data |
0 |
2 |
2 |
4 |
|
2. Training and tuning |
0 |
2 |
2 |
4 |
|
3. Weights and software |
2 |
4 |
10 |
16 |
|
4. Inference interface |
7 |
3 |
5 |
15 |
|
Model development subtotal |
9 |
11 |
19 |
39 |
|
5. Prompt and context |
4 |
22 |
7 |
33 |
|
6. Tools and connectors |
2 |
5 |
8 |
15 |
|
7. Agent identity |
1 |
10 |
5 |
16 |
|
8. Consuming system |
0 |
6 |
6 |
12 |
|
Application runtime subtotal |
7 |
43 |
26 |
76 |
|
Total |
16 |
54 |
45 |
115 |
The technique dates also tell an interesting story:
- 27 of the 39 development entries have been in ATLAS since its first release in 2021
- 61 of the 76 runtime entries were created in 2025 or 2026
So the techniques related to the model development lifecycle (which you buy, basically) have changed little in five years. On the other hand, the techniques related to the application runtime lifecycle (which include the components you likely build and configure) are rapidly growing.
Why might we see more attacks on the application runtime lifecycle in the future? First, consider reach and impact. An attack on the model gets you the model, or its answer. An attack on the system built around the model gets you whatever the agent is permitted to do: move funds, exfiltrate sensitive data, or disrupt services.
Second, consider the relative time and cost of attacking each surface area. Reaching a model developer's training pipeline or querying a model enough times to copy its behavior is expensive. Prompt injection through a poorly controlled interface is often much easier and cheaper.
If your worry about AI risk is that someone will extract your customer list from a model, the evidence says that is still mostly a research result. If your worry is that an agent with a credential will be talked into doing something expensive, the evidence is considerably thicker.
What the Case Studies Tell Us
MITRE scores every case study against the techniques it used and publishes the result as a frequency layer in its ATLAS Navigator. Sorting those scores by lifecycle stage gives another important perspective of where we should focus. The data lets the 23 real incidents (illustrating “realized” techniques) be counted apart from the 50 research exercises (illustrating “demonstrated” techniques).
|
Attack surface stage |
Incidents |
Exercises |
Total |
|---|---|---|---|
|
1. Training data |
2 (9%) |
1 (2%) |
3 (4%) |
|
2. Training and tuning |
1 (4%) |
5 (10%) |
6 (8%) |
|
3. Weights and software |
6 (26%) |
13 (26%) |
19 (26%) |
|
4. Inference interface |
9 (39%) |
21 (42%) |
30 (41%) |
|
Any model development stage |
15 (65%) |
28 (56%) |
43 (59%) |
|
5. Prompt and context |
5 (22%) |
30 (60%) |
35 (48%) |
|
6. Tools and connectors |
2 (9%) |
21 (42%) |
23 (32%) |
|
7. Agent identity |
9 (39%) |
18 (36%) |
27 (37%) |
|
8. Consuming system |
11 (48%) |
39 (78%) |
50 (68%) |
|
Any application runtime stage |
16 (70%) |
47 (94%) |
63 (86%) |
|
Hosting layer (no stage) |
13 (57%) |
19 (38%) |
32 (44%) |
|
Total Case Studies |
23 |
50 |
73 |
Counts are distinct case studies from the ATLAS release dated September 15, 2026. One case study usually touches several stages, so neither the columns nor the stage rows add up. The counts for any model development stage and any application runtime stage reflects the case studies touching at least one stage in that half, which is why each is smaller than the sum of the four rows above it.
A few things stand out from this analysis:
- Runtime-targeted attacks are more prevalent, but not in real life. The case studies touching the application runtime stages outnumber those touching the model development stages by 63 to 43. However, the difference is largely explained by the exercise counts (47 to 28). So, while there are more exercises (i.e., lab-based case studies) impacting the runtime stages, there are roughly the same number of incidents (i.e., real-world attacks).
- The hosting layer matters, a lot. Nearly half (44%) of the case studies involved techniques that target the hosting layer itself. Any AI risk analysis should consider the infrastructure on which the AI-enabled system runs.
- The system that consumes AI output is the most likely target. The consuming system, which includes the software that takes some action based on what the AI model provides, is attacked in over two-thirds of the case studies.
- Training models is not the focus of attacks. Neither the data nor the training and tuning of the models are common targets. Only 9 (12%) case studies reveal this pattern, and none of those case studies involved frontier models like those from Anthropic, OpenAI, Google Gemini, and others.
- The inference interface-related risk is exaggerated. 30 of the 73 case studies touch it, but 18 of those touch it only because the attacker held legitimate access to a model, which is a precondition rather than an attack. Count the studies that reach the stage through an actual technique and the number falls to 12, of which only three are real incidents.
The Controls Are Lagging
ATLAS documents 40 mitigations, or controls. ATLAS pairs fewer of its techniques with a recommended mitigation than MITRE ATT&CK does. In ATLAS, 134 of the 208 entries have at least one mitigation attached, or 64 percent. In ATT&CK's enterprise matrix, 595 of 706 do, or 84 percent. But why?
My first thought was that the AI security field simply has fewer controls to offer, but the data says otherwise. ATLAS publishes 40 mitigations against 208 techniques; ATT&CK publishes 44 against 706. But ATT&CK maps each of its mitigations to more techniques (about 34 on average) than does ATLAS (about 9 on average). Is this just a mitigations mapping gap, or something more fundamental? I don't know the answer to this yet.
My second thought was that the uncovered entries might be the speculative ones, the attacks nobody has pulled off yet. Unfortunately, that is wrong too. Techniques rated Realized, meaning an attacker has actually used them, carry a mitigation 56 percent of the time. Demonstrated and Feasible techniques have an assigned mitigation 72 percent and 71 percent of the time, respectively. The real-life attacks in the catalog are the least likely to have a recommended mitigation.
To be fair, the gap appears to be explained by the age of each technique. Those created before 2025 are covered 75 percent of the time, those created during 2025 69 percent, and those created in 2026 just 46 percent. MITRE catalogs a technique first and maps controls to it later, so the newest material, which is overwhelmingly agentic, is where the coverage is thinnest.
This means that risk professionals will depend more on their own analysis to identify controls that mitigate AI-related risks, especially for more recently discovered techniques.
Put ATLAS In Your Toolbelt
With ATLAS, every technique carries a maturity rating (feasible, demonstrated, realized) and a tag saying what kind of system it applies to. Those two fields let you cut the 208 techniques down to the ones matching your own systems and prioritize those techniques that have been used in real-world attacks. The maturity data is closer to usable frequency data than anything else currently available. Even if you ignore my analysis above, using ATLAS for defending frequency estimates and supporting your controls analysis is why it’s an essential tool for risk professionals.
Next week I'll walk through my four application runtime stages one at a time, naming the types of techniques that land at each and the evidence rating MITRE gives them. This will help you better understand the threats to the parts of the AI system that you likely own, and where to begin assessing and managing the associated risks.




