{"id":209,"date":"2026-09-02T16:26:35","date_gmt":"2026-09-02T07:26:35","guid":{"rendered":"https:\/\/g-gravity.jp\/?post_type=mend-news&#038;p=209"},"modified":"2026-09-17T15:12:46","modified_gmt":"2026-09-17T06:12:46","slug":"sample-mend-news_02","status":"publish","type":"mend-news","link":"https:\/\/g-gravity.jp\/en\/mend-news\/sample-mend-news_02","title":{"rendered":"[OWASP LLM Top 10 2026] AI models will inevitably be deceived. The important thing is, &quot;What is protected when they are deceived?&quot;"},"content":{"rendered":"<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>OWASP&#039;s 2026 LLM Top 10 reveals new trends<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The OWASP GenAI Security Project has released its Top 10 for LLM for 2026. The rankings remain unchanged from last year, with Prompt Injection in first place and Sensitive Information Disclosure in second place, and there appear to be no major changes in the overall rankings.<br>However, a noteworthy change was hidden behind this. The leaders of the OWASP project stated at the outset, &quot;Stop trying to create a model that can&#039;t be fooled, and instead create a system where important things aren&#039;t destroyed when the model is fooled.&quot; What exactly does this mean? For the past two years, the LLM industry has focused on optimizing AI models. However, according to the latest rankings, the focus should be on &quot;risk containment&quot; rather than optimization. The point is to shift the focus from perfect defense to limiting the blast radius of damage.<br>In fact, this assessment seems to accurately reflect the current situation. In this blog, we will analyze the important insights that can be gleaned by carefully interpreting the latest rankings.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Three notable changes<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>1. Discrepancy between practitioners&#039; votes and evidence: misinformation<\/strong>(Misinformation)<strong>Its ranking rose based on evidence.<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Starting with this ranking, evidence data will now be taken into consideration. Practitioner votes account for 751 points (TP3T), and the remaining 25 percent is based on evidence data related to 6,639 real incidents collected from public vulnerability databases and AI damage databases.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The newly adopted 25% evidence data proved most effective in one area: &#039;Misinformation&#039;. While it ranked low in practitioner votes, incident records helped it move up two places in the rankings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The very discrepancy between &quot;practitioner votes and evidence data&quot; contains an important implication. AI engineers and AI security practitioners tend to focus on attacker-initiated risks (injection, leakage, poisoning). However, records of actual incidents objectively show that the most serious incidents were not caused by attackers. &quot;The model simply made a mistake, and subsequent processing accepted it without question&quot;\u2014such simple occurrences are what become reported and result in actual damage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">OWASP explains why this isn&#039;t just a simple output error, but a system-wide failure, as follows: &quot;The model&#039;s output invokes tools, writes code, infers the system&#039;s state, authorizes operations, and connects agents. If the output is wrong at that starting point, it&#039;s no longer just a &#039;wrong output,&#039; but a decision that affects the entire system.&quot;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>2. &quot;Execution&quot; rather than model output is what determines the amount of damage: &quot;Excessive Agency&quot; and &quot;Unbounded Consumption&quot; are causing real damage.<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&quot;Excessive Agency&quot; rose to third place, with both practitioner votes and incident data showing the same trend. This indicates that actual damage is occurring in areas where agents are entrusted with the actual work. &quot;Unbounded Consumption&quot; rose four places in the rankings due to practitioner votes, as resource and cost depletion is now visible in the form of monetary amounts on invoices.<br>The significant rise in these two factors demonstrates that what has the greatest impact on entrepreneurship is not the unsightly output, but the actual actions taken and the money spent.<br>Most of the countermeasures against these risks, known as &quot;least privilege,&quot; have long been discussed in the field of cybersecurity. The concepts of minimizing the scope of privileges, restricting credentials with short validity periods, rate limits and spending caps, and human approval for irreversible operations are the same measures whether it&#039;s AI or humans. These measures are fundamental management methods that AppSec has cultivated for over 20 years and can be directly applied to AI. If there is an element unique to AI, it is that &quot;identifying the workers&quot; itself becomes a task. Naturally, you cannot set the scope of privileges for workers who are outside of management. Only by understanding and designing in a way that can explain what AI components exist in your environment, which ones operate autonomously, what they can access, and who launched them, can the prerequisites for &quot;least privilege&quot; be met.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>3. Categorization and Integration: The agenda shifts from managing &quot;prompts&quot; to managing the &quot;entire context.&quot;<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&quot;System Prompt Leakage&quot; has been renamed to &quot;Hidden Context Exposure.&quot; Also, &quot;Cross-Modal Attack,&quot; which involves attacks hidden in images or audio, has been incorporated into Prompt Injection. &quot;Fine-tuning Subversion&quot; has been incorporated into &quot;Data and Model Poisoning.&quot; &quot;Output Handling&quot; has dropped from 5th to 10th place, but its scope has expanded.<br>OWASP didn&#039;t add new categories, but rather integrated and reorganized existing ones in a rational way. This direction of integration should be understood as a suggestion.<br>The defense now extends beyond just the prompt input field to the entire context that the model loads. This categorization indicates that everything that enters the context window, regardless of its format or who enters it, should be treated as untrusted input.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>OWASP highlights challenges related to the scope of defense.<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The 2026 edition is also notable for clarifying the overall scope of the LLM Application list. The scope of this list is the security risks of LLMs that exist in models incorporated as components of an application. It explicitly states that the moment a model becomes a &quot;self-operating&quot; entity\u2014that is, a model that has tools it can call, carries its memory across sessions, and drives downstream systems\u2014the list to refer to for identifying these risks changes from the Top 10 for LLM Applications to the Aggent Top 10.<br>Just because there are two lists with similar names doesn&#039;t mean you should judge a risk simply based on &quot;which list it&#039;s on.&quot; When identifying application risks related to AI, you need to consider &quot;what the component is capable of.&quot; The following three points are perspectives for specifically asking &quot;what it is capable of.&quot;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Should the state be preserved across sessions?<\/li>\n\n\n\n<li>Can you call up tools or run other systems?<\/li>\n\n\n\n<li>What can you write to, and what lies beyond the destination of those posts? <\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">By checking these three points, you should be able to determine which OWASP list to look at, the extent of the impact, and where containment should be implemented.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Tasks to be done in the next quarter, based on the new OWASP.<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">First, you should start by identifying coding assistants with write permissions to models, agents, MCP servers, and repositories, as well as any AI that the team has set up without informing IT management or security departments. If you can&#039;t actually set permissions and contain the risks based on that information, then it can hardly be called risk management.<br>We identify the components involved with AI and then classify them based on &quot;what they can do.&quot; AI risks cannot be managed by simply organizing which OWASP list they are on, let alone listing components by vendor.<br>Many companies are starting to use AI in their application coding, but how many of them have clearly documented principles such as, &quot;Treat AI-generated suggestions and fixes, including those from our own tools, as unverified input,&quot; and &quot;Do not delegate authority over fix policies to patching agents unless you can take responsibility for their content&quot;?<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We will implement a measurement system to detect when failures proceed without becoming apparent.<\/li>\n\n\n\n<li>The acceptance criteria will be changed from &quot;Has the reproduction code stopped working?&quot; to &quot;Are there still any vulnerable pathways remaining?&quot;<\/li>\n\n\n\n<li>We have our own quality assurance and vulnerability management standards.<\/li>\n\n\n\n<li>We run benchmarks using our own code, targeting bugs that have already been fixed.<\/li>\n\n\n\n<li>Stop making plans based on the average values of other companies.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">In order for AI to truly accelerate application development, incorporating such mechanisms is essential.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Insights from the OWASP LLM Top 10 2026<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">&quot;Even if an AI model fails, we should create a system that prevents it from leading to catastrophic damage.&quot; This is the key takeaway from the new OWASP. So, who can create such a system, and how?<br>At the very least, we can say that &quot;it&#039;s impossible for an AI model to handle the entire mechanism itself.&quot; Just like humans, AI components cannot properly control the scope of their own influence. That said, we shouldn&#039;t leave everything to the vendor of the model that produced the output either. If the vendor creates the models used for inspection and evaluation, the criteria for those judgments will become the same as those of the model that produced the output. In that case, the inspection model will likely overlook any oversights made by the output model.<br>The issue of &quot;the same model being unable to handle control&quot; is not something that can be resolved simply by improving performance. Certainly, the performance of frontier models continues to improve, and a model six months from now might not make the same mistake as today&#039;s model. However, this only reduces the probability of errors; it doesn&#039;t resolve the structural issue that the person auditing the model should not be the creator themselves.<br>Mend.io doesn&#039;t write application code. Our job is purely verification, and we verify the code written by our assistants, whatever it may be that generated.<br>The Mend platform currently offers detection of models, agents, and AI components in your environment, AI-BOM scanning, model risk and agent configuration risk scanning, system prompt hardening, red teaming, and runtime protection within the app or via proxies. We also provide SCA, SAST, container, and IaC scanning of the code itself. AI models are bound to be fooled; whether that becomes a problem depends on what you build around them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">source:<a href=\"https:\/\/genai.owasp.org\/\" data-type=\"link\" data-id=\"https:\/\/genai.owasp.org\/\">OWASP GenAI Security Project<\/a>, Top 10 for LLM Applications 2026 Edition, Top 10 for Agentic Applications 2026 Edition<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>","protected":false},"excerpt":{"rendered":"OWASP\u306e2026 LLM Top 10\u304c\u793a\u3059\u65b0\u305f\u306a\u6f6e\u6d41 OWASP GenAI Security Proj [&hellip;]","protected":false},"author":2,"featured_media":226,"parent":0,"menu_order":0,"template":"","mend-news_category":[15],"class_list":["post-209","mend-news","type-mend-news","status-publish","has-post-thumbnail","hentry","mend-news_category-ai"],"acf":[],"_links":{"self":[{"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/mend-news\/209","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/mend-news"}],"about":[{"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/types\/mend-news"}],"author":[{"embeddable":true,"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/users\/2"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/media\/226"}],"wp:attachment":[{"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/media?parent=209"}],"wp:term":[{"taxonomy":"mend-news_category","embeddable":true,"href":"https:\/\/g-gravity.jp\/en\/wp-json\/wp\/v2\/mend-news_category?post=209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}