OWASP

The first of its kind: how one unbounded request body became a denial-of-service class across every major MCP SDK --- title: "The first of its kind: how one unbounded request body became a denial-of-service class across every major MCP SDK" published: true description: "In April 2026 I reported CVE-2026-39313. Six months later the same flaw had been found and patched in the official Python, Java, Go, Ruby, PHP and TypeScript MCP SDKs. Here is the full class, and why most scanners still cannot see the worst of it." tags: security, mcp, ai, devsecops --- The one-line version In April 2026 I reported CVE-2026-39313, a denial-of-service flaw in an MCP server framework. It was the first CVE of its kind: an unbounded HTTP request body, CWE-770, that lets a single unauthenticated request exhaust a Model Context Protocol server's memory and take it offline. Over the next six months the identical flaw was found and patched in at least nine more MCP projects, including the official Python, Java, Go, Ruby, PHP and TypeScript SDKs. A third-party analysis in July called it "a new vulnerability category." By October two of those CVEs sat in the SDKs that the whole ecosystem is built on, together serving close to half a billion downloads a month. This is the story of that class: how far it spread, who patched and who is still exposed, and why the two worst ones are effectively invisible to the scanners most teams rely on. Every claim below links to a primary source. What the bug actually is MCP servers speak over an HTTP transport: Streamable HTTP, Server-Sent Events, and OAuth routes. To handle a request, the server reads the request body into memory before it does anything else. The flaw is simple and brutal. The read loop has no size limit. It keeps appending chunks to a buffer until the client stops sending or the process dies. There is no authentication in front of it, because the body is read before any auth check runs. So an attacker who can reach the endpoint sends one large POST, or a slow endless stream, and the server allocates memory until it is killed by the OOM reaper. One request, one dead server, no credentials required. This is CWE-770, allocation of resources without limits. It is not exotic. That is exactly why it turned up everywhere. CVE-2026-39313: the first one The first public advisory of this class was in mcp-framework, a popular TypeScript framework for building MCP servers. In versions 0.2.21 and below, the readRequestBody() function in the HTTP transport concatenated body chunks into a string with no cap. The framework even had a maxMessageSize configuration value, but readRequestBody() never referenced it. The limit existed on paper and was never enforced in code. A single large POST to /mcp crashed the server. I reported it, it was fixed in 0.2.22, and the advisory published on 16 April 2026. - Advisory (GitHub, GHSA-353c-v8x9-v7c3): github.com/QuantGeekDev/mcp-framework/security/ad… - CVSS 4.0 8.7 (High), CWE-770 At the time it looked like one framework's mistake. It was not. It was not a bug. It was a pattern. Here is the timeline of the same unbounded-body denial-of-service turning up across the MCP ecosystem, as advisories and patches landed through 2026: - 16 April — mcp-framework, CVE-2026-39313 (the first). - April to June — Google's MCP Toolbox for Databases had the identical io.ReadAll with no limit, reported four days after my advisory and fixed in June. No advisory, no CVE. - July — the official Go SDK added a request-body cap, shipped as OOM protection in its release notes, with no advisory and no CVE. - 30 July — the Ruby SDK, CVE-2026-67432. A 512 MB POST grew a worker from 44 MB to 1.66 GB. Fixed with a 4 MiB default cap. - 19 August — the PHP SDK, CVE-2026-53965, the client-side version: an unbounded SSE buffer. The Java SDK shipped a matching client-side SSE fix the same day. - 16 September — rmcp, the Rust SDK, CVE-2026-63128. - 11 October — mcp-go, CVE-2026-108859. - 14 October — the official Python SDK, CVE-2026-109527, and the official Java SDK, CVE-2026-108395. Both credited to me among the reporters. - 20 and 27 October — the official TypeScript SDK, CVE-2026-111768 and a follow-up advisory. Nine-plus independent projects. The same root cause. The same fix every single time: a default body-size cap, almost always 4 MiB. In July an independent write-up looked at this and called CVE-2026-39313 the first documented denial-of-service against the MCP HTTP transport and "a new vulnerability category." That is the right framing. This was not a run of unrelated bugs. It was one design gap in how MCP transports read input, replicated across every language the protocol ships in. The two that carry the protocol Most of the class lived in frameworks and bindings. Two of them did not. They were in the reference SDKs that nearly every MCP deployment imports. CVE-2026-109527, the official Python SDK (mcp). In versions before 1.29.1, and in the 2.0 pre-releases through 2.0.1, the Streamable HTTP session manager, the SSE message endpoint, and the OAuth authorization-server routes all buffered complete request bodies with no maximum size. An unauthenticated remote client could send a large declared or streamed body that consumed several times its own size in process memory, and repeat it until the server died. - Advisory (GHSA-fmmv-w9g8-j3gc): github.com/modelcontextprotocol/python-sdk/securi… - Fixed in 1.29.1. CVSS 7.5 (High), CWE-770. - The mcp package serves over 235 million downloads a month on PyPI. CVE-2026-108395, the official Java SDK. Unbounded request-body reads in the mcp-core HTTP server transports, same class, same impact. - Advisory (GHSA-4x8c-5vv7-973f): github.com/modelcontextprotocol/java-sdk/security… - CVSS 7.5 (High), CWE-770. For scale: the official TypeScript SDK pulls around 257 million downloads a month, the Python SDK 235 million. When the flaw sits in the SDK rather than a single server, every server built on that SDK inherits it the moment it exposes an HTTP transport. Who patched, -> Red Hat rated CVE-2026-109527 Important, assessed 44 of its products, and issued three errata: - RHSA-2026:92510 (OpenShift AI 3.4), RHSA-2026:92666 and RHSA-2026:92667 (Satellite 6.19 and 6.18). - Of the 44 products assessed, 35 are affected, 6 not affected, 3 marked will-not-fix. - access.redhat.com Debian tracks the same CVE against python-mcp at severity grave, and at the time of writing the package in unstable is still unfixed. - bugs.debian.org Downstream products started bumping their pins through October: OpenMetadata merged its upgrades, Red Hat's Open Data Hub opened fix PRs, and several hundred automated "bump mcp" pull requests have landed across GitHub since the advisories went live. So the serious operators with security teams are moving. The question is everyone else. The specification still does not require this For all the patching, the root cause has never been closed at the design level. As of early November 2026 the MCP security best-practices document still contains no guidance on request-body size limits. Every SDK solved it independently, with its own default, because the specification never told them to. That is why the same fix had to be rediscovered six, seven, eight times over. The MCP server security guidance that does exist, including the National Security Agency and CISA information sheet on MCP, discusses resource-exhaustion denial of service as a general risk but points implementers at no concrete control for this. The one place that could stop the whole class at the design level still does not mention it. The blind spot: most scanners cannot see the worst two This is current as of 3 November 2026, and it may change the day the databases catch up. Check it yourself. CVE-2026-39313, the npm one, has a full global GitHub advisory. It is in OSV, it is in npm audit, and a scan of a vulnerable project flags it. The two SDK CVEs do not behave the same way. CVE-2026-109527 and CVE-2026-108395 exist only as repository-level advisories on the Python and Java SDK projects. They were never promoted into the global GitHub Advisory Database. The downstream effect is a chain of blindness: - They are absent from the global GitHub advisory mirror, so GitHub Dependabot does not alert on them. - They are absent from OSV, so osv-scanner and everything built on OSV misses them. Querying a vulnerable mcp version returns the other 2026 CVEs but not this one. - They have no record in the PyPA advisory database, so pip-audit finds nothing. - NVD lists both as "Awaiting Analysis" with no version data, so even NVD-driven scanners have no range to match on. The practical result: a server running a vulnerable mcp package can be scanned by pip-audit, osv-scanner, Dependabot, or a popular MCP-specific scanner that wraps pip-audit, and come back clean for the denial-of-service flaw. The patch shipped, the advisory was written, and the machine-readable plumbing that is supposed to warn people never carried it. The enterprise distribution trackers, Red Hat, Debian and SUSE, do carry it, because they ingest from their own CNA and NVD feeds. Red Hat has even published a machine-readable VEX file for it. But the developer-facing supply-chain tools that most teams actually run do not. If you depend on pip-audit or Dependabot to tell you about this, right now they will not. What an attacker gets, and who is at risk You are exposed if you run an MCP server that: - exposes an HTTP-based transport (Streamable HTTP or SSE), as opposed to stdio only, and - is reachable by any client you do not fully trust, which includes the public internet, a shared internal network, or a browser-reachable origin, and - runs a vulnerable version of the framework or SDK below. The attack needs no authentication, no credentials, and no user interaction. One oversized or endless request crashes the process. Repeat it and the service stays down. It is trivially scriptable. There is no data theft here, the impact is pure availability, but for an agent backend that other systems depend on, taking it offline on demand is the whole game. Stdio-only servers and deployments already sitting behind a reverse proxy with its own request-body limit are not exposed in the same way. That proxy limit is also your fastest mitigation while you patch. The fix Upgrade. The fixed versions: - mcp-framework (npm): 0.2.22 or later. - MCP Python SDK (mcp on PyPI): 1.29.1 or later. The fix adds max_request_body_size with a DEFAULT_MAX_REQUEST_BODY_SIZE of 4 MiB in the transport security layer. - MCP Java SDK: the patched mcp-core release. - MCP Go SDK: v1.7.0 or later, which enforces DefaultMaxRequestBodyBytes of 4 MiB. - Ruby SDK (mcp gem): 0.23.0 or later. - PHP SDK, Rust rmcp, mcp-go, TypeScript SDK: upgrade to the latest; all shipped the same body-size cap through the autumn. If you cannot upgrade immediately, put a reverse proxy in front with a client body limit (for example nginx client_max_body_size), and make sure your HTTP transport is not reachable from untrusted networks. Where your SDK exposes a configurable limit, set it explicitly rather than trusting the default, especially if you legitimately handle large payloads. How to find it in your estate - Inventory every MCP server you run and whether it exposes an HTTP transport or stdio only. - For Python, do not trust pip-audit alone for this one today. Check the installed mcp version directly: anything below 1.29.1 is vulnerable. - For npm, npm audit will catch mcp-framework below 0.2.22. - For Java and Go, check the SDK version against the fixed releases above, because the advisories are not yet in the global databases your SCA tooling reads. - Grep your own HTTP handlers for the pattern itself: a body read with no length cap, io.ReadAll(r.Body) in Go, request.body.read with no bound in Ruby, a string += chunk loop in Node, before any size check. If you wrote a custom transport, you may have reproduced the class yourself. The lesson The MCP ecosystem grew faster than its security guidance. A single gap in how transports read input became a denial-of-service class that reached every major SDK in the protocol, and the one control that prevents it is still not written into the specification. The fixes exist now. The specification guidance still does not. And for the two CVEs in the core SDKs, the automated warning system that the rest of us depend on is, as of today, silent. If you run MCP in production, do not wait for a scanner to tell you. Check the versions yourself. References - CVE-2026-39313, mcp-framework: github.com/QuantGeekDev/mcp-framework/security/ad… - CVE-2026-109527, MCP Python SDK: github.com/modelcontextprotocol/python-sdk/securi… - CVE-2026-108395, MCP Java SDK: github.com/modelcontextprotocol/java-sdk/security… - Red Hat, CVE-2026-109527: access.redhat.com - Debian tracker (grave), python-mcp: bugs.debian.org - OWASP AISVS, MCP transport payload-size control citing CVE-2026-39313: github.com/OWASP/AISVS/blob/main/1.0/research/cha… --- Raza Sharif, CyberSecAI Ltd. Reported CVE-2026-39313 and is credited on CVE-2026-109527 and CVE-2026-108395. All vulnerabilities were handled through coordinated disclosure and are fixed in the versions listed above. Contact: contact@agentsign.dev.

Debating How to Threat Model Terraform Infrastructure Code: Simon Kang and Srdan Reljic at Global AppSec USA 2026 Terraform defines a company's cloud infrastructure in code, so one pull request can open an S3 bucket to the internet or widen a firewall rule before anyone reviews the design. Because the infrastructure exists as code, tools can read each Terraform change and generate a threat model from it automatically. Teams building those tools have to decide how much of the analysis to hand to an LLM. At SecretCon in June 2026, Srdan Reljic and Raj Annaladasu presented an AI agent that reads Terraform, flags risky settings such as open S3 buckets, then maps each finding to MITRE ATT&CK. Annaladasu has since moved every security decision in his pipeline to deterministic code. LLMs now only explain findings, map them to frameworks and suggest fixes. In their Global AppSec USA 2026 debate, "OWASP by Design: Debate/panel for Terraform", on Thursday, November 5 at 12:00 p.m. in Bayview A, Simon Kang and Srdan Reljic hold a 15-minute discussion on Terraform threat modeling. Speakers: - Simon Kang, Principal TPM, Enterprise AI, Snowflake - Srdan Reljic, Associate Partner, IBM Consulting, Americas The debate follows Annaladasu's talk, "From Terraform Plan to Attack Paths: PyTM Threat Modeling in CI/CD with LLMs on a Leash," in the same room at 11:15 a.m. Simon Kang is Principal Technical Program Manager for Enterprise AI at Snowflake and describes himself as a cybersecurity educator. His work connects threat modeling and AI governance with controls that developers can build, which reviewers can then assess. His interests include agent identity, retrieval authorization and tool-call controls. He co-presents "Threat Modeling AI Agents That Can Act: Identity, Retrieval, and Tool-Call Controls" with Annaladasu on Thursday at 3:15 p.m. in Bayview A. Srdan Reljic is an Associate Partner at IBM Consulting, Americas. His work applies cloud-native and open-source technology to build security into every layer of a system. His interests include secure developer enablement, platform engineering and GenAI security. He previously worked as Business Development Executive for Cloud Native Build Americas at IBM. The SecretCon talk he co-presented describes an agent that turns Terraform into a resource graph and feeds code-level fixes back into CI/CD. It also maps findings to CIS Benchmarks. To go deeper on the topic, start here: - SecretCon 2026: AI-Driven Terraform Threat Modeling (Reljic and Annaladasu) - OWASP pytm on GitHub - OWASP Developer Guide: pytm - Checkov on GitHub Cloud and platform teams deciding how far to trust AI in their infrastructure security reviews will hear two views from people who build these systems. Would you let an LLM decide whether a Terraform change is risky, or only explain a decision a deterministic tool already made?

Turning Terraform Plans Into Attack Paths Without Letting an LLM Make the Security Calls: Raj Annaladasu at Global AppSec USA 2026 Infrastructure-as-code scanners such as Checkov and tfsec flag individual misconfigurations in Terraform, but they do not show how those misconfigurations connect into an attack path. Raj Annaladasu built a pipeline that turns a Terraform plan into an executable OWASP pytm threat model on every pull request. Pytm is OWASP's framework for writing threat models as Python code. His first version let an LLM generate the threat model itself. That cost the repeatability and traceability a CI/CD pipeline needs, since the same plan could yield different results on different runs. In his Global AppSec USA 2026 talk, "From Terraform Plan to Attack Paths: PyTM Threat Modeling in CI/CD with LLMs on a Leash", on Thursday, November 5 at 11:15 a.m. in Bayview A, Raj Annaladasu walks through the current design. Speaker: Raj Annaladasu, Staff Customer Success Engineer, Palo Alto Networks The pipeline builds a typed graph of the Terraform resources and generates data flow diagrams from it. It also infers IAM relationships that Terraform never states explicitly. Every security decision in the current design is deterministic. LLMs only explain findings, map them to security frameworks, suggest fixes and document attack paths. Every AI annotation must point to real evidence in the plan, or the pipeline drops it. The session compares the pipeline live against Checkov, tfsec and a pytm-only approach. It includes examples where AI helped, along with examples where it did not. It also demonstrates prompt injection hidden in Terraform comments steering AI tooling, then the safeguards against it. Attendees leave with an architecture they can build. The session repeats as a hands-on OWASP Project Demo Lab on Friday, November 6 at 1:15 p.m. in Golden Gate. A 15-minute debate on Terraform with Simon Kang and Srdan Reljic follows this talk in Bayview A at 12:00 p.m. Raj Annaladasu is a Staff Customer Success Engineer at Palo Alto Networks, focused on application, cloud-native, software supply chain and AI security. His work includes risk assessments, static analysis and software composition analysis. With Srdan Reljic of IBM Consulting, he presented "AI-Driven Terraform Threat Modeling" at SecretCon in June 2026. That earlier version mapped Terraform findings to MITRE ATT&CK and CIS Benchmarks. He also co-presents "Threat Modeling AI Agents That Can Act: Identity, Retrieval, and Tool-Call Controls" with Simon Kang on Thursday at 3:15 p.m. He then joins a debate on threat modeling agents at 4:00 p.m. The OWASP Developer Guide explains how pytm turns a model written in Python into data flow diagrams, sequence diagrams and threat reports. Aqua Security has folded tfsec into Trivy and published a migration guide. To go deeper on the topic, start here: - SecretCon 2026: AI-Driven Terraform Threat Modeling (Reljic and Annaladasu) - OWASP pytm on GitHub - OWASP Developer Guide: pytm - Checkov on GitHub - tfsec to Trivy migration guide Cloud security engineers who already scan Terraform in CI will see how to connect isolated findings into attack paths. Does your Terraform scanning show how two low-severity findings combine, or only list them separately?

A Patch-to-Score CTF You Can Run From a Laptop With Docker Compose: Jon McCoy, Brandon Lachterman and Diego Cotelo at Global AppSec USA 2026 The first OWASP CTF at DEF CON 34 in August 2026 drew 60 players, who submitted more than 2,300 patches against six vulnerable OWASP training applications. Players scored by fixing vulnerabilities rather than exploiting them. Each patch went in as a GitHub pull request, and an automated test judged whether the fix held. Running a hands-on security event usually means cloud accounts, ongoing bills and custom scoring infrastructure. OWASP CTF in a Box packages the platform behind that event so that any chapter, class or AppSec team can host its own. In their Global AppSec USA 2026 demo lab, "OWASP CTF in a Box: Host Your Own Security CTF, from a Laptop to the Cloud", on Thursday, November 5 at 10:30 a.m. in Golden Gate, Jon McCoy, Brandon Lachterman and Diego Cotelo walk through the full life of an event on the platform. Speakers: - Jon McCoy, Principal Security Architect & Offensive Engineering Specialist, DigitalBodyGuard - Brandon Lachterman - Diego Cotelo, Head of Infrastructure and Security, Crowder The platform turns a free GitHub organization into a complete event, with team registration, a live leaderboard, an organizer console and automated scoring. It runs on a single laptop or VPS with Docker Compose, or deploys to fly.io or AWS as the event grows. Its modules include Secure Development, where participants fork a deliberately vulnerable app to score by fixing it, along with quiz and Jeopardy-style challenge boards. AI security challenges hosted elsewhere can feed the same scoreboard. The demo covers launching an event in minutes, then running it live: opening registration, scheduling the scoring window, managing teams or hints, and freezing scoring. It ends with customizing branding and contributing new challenge modules. Jon McCoy is Principal Security Architect & Offensive Engineering Specialist at DigitalBodyGuard, which he founded. He built some of the first tools for reverse engineering .NET applications and presented on attacking them at DEF CON 18 in 2010. Diego Cotelo is Head of Infrastructure and Security at Crowder in Montevideo, Uruguay. He has more than 15 years of experience across infrastructure, security consulting and cloud architecture. He previously worked as a Staff Engineer at Fingerprint and a Cloud Architect at Caylent. Cotelo co-built the OWASP CTF at DEF CON 34. He co-leads CTF in a Box with McCoy and Chris Maenner. The OWASP project page lists CTF in a Box as an Incubator project written in TypeScript, with 321 challenges across six training targets. The DEF CON 34 event site breaks those down by app, from 110 challenges in VulnerableApp to 9 in VAmPI. The how-to-play guide walks through the scoring loop with a real example: a Juice Shop SQL injection in routes/login.ts, fixed with bind parameters. To go deeper on the topic, start here: - OWASP CTF in a Box on GitHub - OWASP CTF in a Box project page - OWASP CTF at DEF CON 34: challenges and format - How the patch-to-score CTF works OWASP chapter leaders, educators and AppSec teams who want to run a defense-focused CTF without building scoring infrastructure are the audience for this demo. Would your developers learn more from a CTF where they fix bugs or one where they exploit them?

From 200 Container CVEs to the Few Dockerfile Lines Worth Fixing: Advait Patel at Global AppSec USA 2026 A container image scan typically returns more than 200 CVEs, most of them noise, with no indication of which few Dockerfile lines a developer should change. According to Advait Patel's session description, those reports get ignored. Someone spends half a day looking up each CVE, then files the rest as accepted risk. OWASP DockSec, which Patel created, sits on top of existing scanners rather than replacing them. It runs Trivy, Hadolint and Docker Scout locally. An LLM then correlates the results, removes duplicates, then ranks what matters for the specific setup. In his Global AppSec USA 2026 demo lab, "DockSec: Closing the Gap Between What Your Scanner Finds and What You Actually Fix", on Thursday, November 5 at 10:30 a.m. in Golden Gate, Advait Patel runs real Dockerfiles through the tool live. Speaker: Advait Patel, Senior Site Reliability Engineer, Broadcom The demo includes Dockerfiles that pass standard scanning clean. For each issue DockSec surfaces, it gives a plain-English explanation along with the exact line-level fix. DockSec produces a 0 to 100 security score for tracking posture over time. It exports HTML, PDF, JSON, CSV or Markdown reports, so security and development teams share one artifact. Only scan metadata leaves the machine; image contents never do. DockSec works with OpenAI, Anthropic or Google Gemini models, or fully locally through Ollama with no API key, and it runs in CI/CD on every build. Attendees can bring their own Dockerfiles. Patel also gives a 30-minute DockSec project showcase on Thursday at 2:45 p.m. in Seacliff CD. Advait Patel is a Senior Site Reliability Engineer at Broadcom in Chicago and a Docker Captain. He has more than eight years of experience in cloud-native architecture, container security and DevSecOps. He is a Google Developer Expert for Google Cloud and AI. He is a founding member of the OWASP AI Vulnerability Scoring System (AIVSS) project. He has co-authored two Springer books on Google Cloud security, along with Cloud Security Alliance guidance on securing LLM workloads. His speaking credits include QCon AI Boston, SANS CloudSecNext, OWASP SnowFROC, PlatformCon and the Linux Security Summit North America. The OWASP DockSec project page shows it reached OWASP Lab status in June 2026, with Arkadii Yakovets as co-leader. It also outputs SARIF and CycloneDX SBOM files. Secrets are redacted before any content reaches an AI provider. SC World reported about 17,000 PyPI downloads. It also described one startup that cut production CVEs by 78 percent, from about 22 to about 5 per month, while triage time per image fell from about 45 minutes to about 6. To go deeper on the topic, start here: - ISACA Podcast: Your Containers Are Probably Full of Holes: Fixing Docker Security with AI (Advait Patel) - OWASP/DockSec on GitHub - Help Net Security interview with Advait Patel on DockSec - SC World: DockSec adoption and triage-time results - docksec on PyPI Platform engineers who own container images, and who are tired of untriaged scanner output, are the audience for this demo. How many open container CVEs does your team currently carry as accepted risk?

Turning AI Risks Into Testable Security Requirements With OWASP AISVS: Rico Komenda and Jim Manico at Global AppSec USA 2026 OWASP's AI Security Verification Standard reached version 1.0 in June 2026, with 12 chapters of testable requirements for applications built on models, LLMs and AI agents. Most AI security guidance describes risks at a high level. Engineering teams deciding whether an AI-enabled application is secure enough to ship need requirements they can check one by one. AISVS adapts the idea behind OWASP's Application Security Verification Standard (ASVS) for web apps: security should be verifiable and measurable. It turns AI risks into requirements a team can use in design review, testing, procurement and production readiness. In their Global AppSec USA 2026 talk, "OWASP AISVS: Bringing Order to AI Security Chaos", on Thursday, November 5 at 10:30 a.m. in Seacliff CD, Rico Komenda and Jim Manico explain how the standard was built, along with how teams can apply it. Speakers: - Rico Komenda, Senior Product Security Engineer - Jim Manico, Founder, Manicode Security The speakers argue that security practice still assumes deterministic software, while AI systems produce probabilistic output and make decisions on the fly. AISVS keeps the parts of conventional secure development that still apply, then adds new verification categories where they do not. The new categories cover prompts, models, agents, training data, retrieval-augmented generation (RAG), tool use, output handling and AI governance. The talk also shares lessons from building the standard, including where it differs from traditional AppSec. The AISVS project page lists 12 chapters, from training data integrity through MCP security to monitoring and anomaly detection. Requirements fall into three levels: a baseline, a level for production or customer-facing apps, and a level for safety-critical or regulated systems. Rico Komenda is a Senior Product Security Engineer who works on the attack surface of AI-integrated systems and secure agentic development. He is one of the AISVS project leaders, along with founder Jim Manico, Otto Sulin and Russ Memisyazici. Komenda contributes to the OWASP GenAI Security Project. He previously worked as a Senior Security Consultant at adesso SE. At Global AppSec EU 2025 in Barcelona, he co-presented "Builders and Breakers: A Collaborative Look at Securing LLM-Integrated Apps," taking the red team side with live data poisoning demos. His other talks include "Pentesting LLM-integrated applications" at JavaLand 2025. Jim Manico founded Manicode Security, which trains developers in secure coding. He founded AISVS and wrote "Iron-Clad Java: Building Secure Web Applications." He also co-leads OWASP ASVS along with the OWASP Cheat Sheet Series. Komenda also presents "Security Chaos Engineering for AI Application Architectures" on Thursday at 1:45 p.m. The two speakers then run a hands-on AISVS assessment workshop on Friday, November 6 at 10:30 a.m. in Golden Gate. The pair first presented AISVS together in a lightning talk at Global AppSec EU 2026 in Vienna, before version 1.0 shipped. LRQA Cyber Labs has since published a chapter-by-chapter overview of the 1.0 release. To go deeper on the topic, start here: - OWASP AISVS GitHub repository - OWASP AISVS project page - LRQA Cyber Labs: OWASP AISVS 1.0 chapter-by-chapter overview - VirtusLab: OWASP AISVS 1.0 walkthrough - ITSPmagazine interview with Jim Manico on AISVS AppSec teams asked to sign off on AI features, without a checklist that fits them, will find one here. What would your team check first before approving an AI feature for production?

That the American Osteopathic College of Radiology [aocr.org](aocr.org) have such an incredible interest in our main Cornucopia repository (owasp/cornucopia) on Github is a mystery to me. Does anyone care to elaborate? #appsec #security #osint

AI Security Testing 102: Sensitive Data Leak, Input Leakage, and Prompt Disclosure This is the second post in a new series on AI application security testing, following the structure of the OWASP AI Testing Guide's AI Application Testing category (AITG-APP). The previous post covered prompt injection: testing whether attacker-controlled input can change an application's behavior in ways that violate its intended instructions or security constraints. This post focuses on a related but distinct question: whether an AI application exposes information that should remain protected. We're covering three tests from the guide. Although all three involve information being exposed in an unintended way, they target different sources of information and different security boundaries. | Test | The question it asks | Where the information comes from | |---|---|---| | AITG-APP-03 — Sensitive data leak | Can the application expose confidential information through its model, memory, training data, or connected tools? | Model, memory, training data, retrieved documents, connected tools | | AITG-APP-04 — Input leakage | Can information supplied by a user be unnecessarily reproduced or exposed through the application? | The user's own input | | AITG-APP-07 — Prompt disclosure | Can the application reveal its system or application instructions? | The application's instructions | The distinction matters because the same apparent symptom can have very different causes. Effective testing therefore needs to establish where the information came from, whether the disclosure was unauthorized, and which security control should have prevented it. --- Sensitive Data Leak (AITG-APP-03) Sensitive data leakage occurs when an AI application exposes confidential information that should not be available to the requesting user or should not leave a protected system boundary. The source of the information can vary. It may come from training data, conversation memory, retrieved documents, application context, databases, or connected tools. The testing approach therefore depends on what information the application can access and which users are authorized to see it. A useful starting point is to identify what the application should protect and where that information enters the system. Testing can then examine whether the model or surrounding application exposes it through direct requests, contextual tasks, or unintended information disclosure. Attack Patterns Worth Knowing Several approaches can help testers identify whether sensitive information is exposed through an AI application. Direct and contextual requests Simply asking the model for credentials or confidential information is an obvious starting point. A more contextual test might ask the model to summarize internal communications or retrieve information from a document that happens to contain sensitive values. The distinction matters because an application may refuse a direct request for a password while still exposing the same information through an apparently legitimate task, such as summarization or document analysis. Persona and pretext framing A request might frame the interaction as a developer debugging the application, an administrator investigating an error, or a researcher examining a system's configuration. These scenarios can help determine whether the application relies too heavily on the model's interpretation of the user's claimed role or purpose. Authorization should be enforced by the application rather than inferred from a persuasive explanation in the prompt. Tool and application reconnaissance For agentic systems, testers can examine whether the model reveals available tools, sub-agents, tool parameters, or other operational details. This information is not necessarily confidential, and revealing a tool description is not automatically a vulnerability. However, unnecessary disclosure can help an attacker understand the application's capabilities and construct more targeted follow-up attacks. Testing should establish whether the disclosed information violates a confidentiality requirement or meaningfully weakens a security boundary. Implementation details and error disclosure Requests for stack traces, API endpoints, framework details, internal configuration, or recent errors can reveal information about the application's implementation. Individually, some of these details may be harmless. Others can expose sensitive values, internal infrastructure, or information that should be restricted to administrators. The security impact depends on what is disclosed and who can access it. The Unifying Theme A sensitive data leak is not always a model revealing a password. Information about internal systems, retrieved records, or connected tools can also be relevant when its disclosure violates the application's intended security constraints. > Distinguish actual disclosure from speculation. A model describing a hypothetical environment does not prove that it has access to the corresponding credentials, files, or infrastructure. --- Input Leakage (AITG-APP-04) Input leakage examines whether information supplied to an AI application is exposed or reused in ways that violate the application's intended privacy or security boundaries. Unlike a sensitive data leak, where the information may originate from a protected data source or connected system, this test starts with information provided through the input channel itself. A practical test might use a synthetic API key, a fake password, or a clearly marked confidential string. The tester then checks whether the application: - reproduces that value in a later response, - exposes it to another user or session, or - passes it to a component that should not receive it. The important question is not simply whether the model can repeat the value. It is whether the application handles that value in accordance with its security requirements. Common Test Patterns Repeated-input testing Provide a synthetic sensitive value and check whether the application reproduces it when asked directly or in a later turn. Where relevant, test session boundaries and whether information can appear in another user's context. This helps distinguish ordinary repetition within a conversation from unintended exposure across users or sessions. Normalization and obfuscation Represent test values using alternative character encodings or Unicode variants, then evaluate whether the application's detection and redaction controls recognize the value. For example, a test might use Unicode full-width digits instead of standard ASCII digits. A filter that relies on a narrowly defined character pattern may handle the two representations differently, while a downstream component may normalize them to the same value. This is not a guaranteed bypass. The result depends on how inputs are normalized, inspected, and processed throughout the application. Contextual variation Change the surrounding request while keeping the underlying sensitive value the same. For example, compare a direct request to reproduce a value with a request to summarize, transform, or explain the message containing it. The objective is to determine whether the application's protections depend too heavily on specific keywords or request formats. Affix and formatting variations Add unrelated text immediately before or after a synthetic sensitive value, or place the value within a larger structured input. These variations can help identify filters that rely on particular positions, delimiters, or surrounding context. Their effectiveness depends on the implementation being tested. Combined test cases Combine several input variations to evaluate whether the application's controls continue to work when multiple techniques are used together. For example, a test might combine Unicode variations with additional surrounding text and a contextual request. This can help establish whether different defensive layers work together or whether one layer depends on another recognizing a particular representation. Testing Ground Rules These tests should use synthetic values rather than real credentials or personal information. They should also distinguish between the model's response, application logs, downstream tool calls, and any cross-session exposure being tested. > A clean result has limits. A successful output test only establishes what happened in the observed interaction. It does not establish whether the provider retains inputs, whether those inputs are used for training, or how they are handled in application logs. Those questions require separate examination of the system's data-handling configuration and policies. --- Prompt Disclosure (AITG-APP-07) Prompt disclosure tests whether an AI application reveals system or application instructions that are intended to remain confidential. These instructions may contain proprietary workflows, internal policy details, tool descriptions, or other implementation information. Their disclosure can help an attacker understand how the application is designed and identify additional attack paths. However, a system prompt should not be treated as a security boundary in its own right. Secrets should not be embedded in instructions, and authorization should be enforced by the application rather than relying on the model to follow a prompt. Testing should therefore establish both whether protected instructions can be reconstructed and whether their disclosure creates a meaningful security or operational impact. Techniques Worth Testing Direct and oblique requests Simply asking the model to reveal its system prompt is an obvious starting point. More indirect requests might ask it to explain its guidelines, describe its operating instructions, or summarize the rules governing its behavior. Oblique phrasing can sometimes produce different results from a direct request, depending on the model and the application's safeguards. Testers should evaluate both approaches rather than assuming that blocking one phrase prevents disclosure. Context and clarification framing A request might claim that the model's behavior is confusing and ask it to quote the instructions responsible for that behavior. This tests whether the model treats a request framed as clarification differently from an explicit request for confidential instructions. The important question is whether the application exposes information that its security requirements say should remain protected. Partial completion and reconstruction A tester can examine whether supplying a suspected prefix of an instruction causes the model to continue with confidential text. Language models generate text based on their learned behavior and the context they receive, including instructions and learned refusal behavior. Prefix completion may therefore reveal information in some configurations, but it does not guarantee disclosure. Testing should also distinguish between the model producing a plausible continuation and actually reproducing the application's real system prompt. Incremental extraction Instead of requesting an entire prompt at once, a tester can examine whether the application reveals protected instructions in smaller pieces. For example, successive requests might ask for individual instructions, a summary of particular rules, or additional details about previously disclosed text. The objective is to determine whether restrictions that apply to a single large request remain effective across multiple interactions. Adversarial suffixes (a related research area) Adversarial suffixes are specially constructed text appended to a prompt to influence a model's behavior. Some research has demonstrated that such techniques can induce undesirable outputs or weaken refusal behavior in particular configurations. However, causing a model to produce an unwanted response is not the same as demonstrating system-prompt extraction. A prompt-disclosure finding requires evidence that protected instructions were actually revealed or reconstructed. The broader testing principle is to vary the request format and interaction pattern rather than assuming that one refusal demonstrates that the system prompt is protected. --- What Makes These Tests Difficult The three tests share an important challenge: the model's visible response does not necessarily tell you where the information originated or which control failed. Model access versus application access A model may generate a plausible-looking secret without actually having access to it. Conversely, an application may retrieve confidential information and pass it into the model's context even when the model would not otherwise know that information. Testers should establish whether a disclosed value is real, whether the application had access to it, and how it reached the response. Conversation context and session boundaries A model may repeat information because it remains available in the current conversation context. That behavior is different from exposing information across isolated sessions or retaining it beyond the intended lifetime. Testing should account for conversation history, application memory, retrieval systems, and any explicit persistence mechanisms. Multiple defensive layers An application may use input filters, output classifiers, retrieval permissions, redaction, and model-level safeguards. A successful or unsuccessful request may reflect the combined behavior of these components rather than the underlying model alone. Where practical, testers should inspect application logs, retrieval traces, and tool calls to identify which control accepted or rejected the request. Configuration and repeatability Results can depend on the model, application instructions, available tools, conversation history, and sampling configuration. A single successful or unsuccessful attempt is not necessarily conclusive. Relevant tests should be repeated under representative configurations, with enough information recorded to make the results reproducible. > These challenges make observability important. Without knowing what information entered the model's context and what happened after generation, it can be difficult to distinguish an actual disclosure from a plausible but unsupported response. --- What Counts as a Finding Across these three tests, a vulnerability is confirmed when the application exposes information in a way that violates its intended security or privacy constraints. Examples include: - The application reveals credentials, API keys, personal data, or confidential records to a user who should not have access to them. - Information supplied in one context becomes accessible in another context where it should remain isolated. - Sensitive input is passed to a downstream tool, service, or logging component that is not authorized to receive it. - The application discloses protected system instructions, and that disclosure violates an established confidentiality requirement or creates a demonstrable security risk. The distinction between disclosure and impact matters. A model repeating a synthetic test string to the same user is not necessarily a vulnerability. Likewise, revealing a system prompt is not automatically a security incident if the prompt contains no protected information and its disclosure does not violate a meaningful requirement. For each finding, establish: 1. What information was exposed 2. Where it originated 3. Who could access it 4. Which security boundary failed For input leakage specifically, an unsuccessful attempt to reproduce a sensitive value does not prove that the application has no retention or logging risks. Those require separate tests of the relevant data-handling mechanisms. --- Remediation, in Short These tests point toward several overlapping defensive principles. Protect sensitive data at the application layer Apply access controls before retrieving or exposing confidential information. Enforce authorization outside the model, minimize the data included in model context, and redact sensitive values from outputs where appropriate. Review training and retrieval pipelines for inadvertently exposed secrets, and ensure that the model receives only the information required for its task. Isolate sensitive user input Define how sensitive input should be handled throughout its lifecycle, including model context, conversation history, application logs, and downstream tool calls. Use synthetic test values to verify that redaction, normalization, and access controls behave as intended. Where sensitive information should not persist, enforce that requirement through application and data-retention controls rather than relying on the model to forget it. Treat system prompts as instructions, not secrets storage Avoid embedding credentials, API keys, or other sensitive values in system prompts. Limit unnecessary internal implementation details, and test whether confidential instructions can be reconstructed through direct requests, partial extraction, or contextual framing. Prompt design and model-level safeguards can reduce unwanted disclosure, but they should not be the only controls protecting confidential information. Enforce least privilege and session isolation Restrict access to tools, documents, databases, and other resources according to the requesting user's permissions. Ensure that conversation state and retrieved information cannot cross user or session boundaries unintentionally. Monitor information flows Where practical, log and monitor access to sensitive data, model inputs and outputs, retrieval operations, and tool calls. Protect those logs appropriately and use them to investigate suspected disclosure without creating additional copies of sensitive information. > The central principle: enforce confidentiality and authorization in the surrounding application. A model's willingness to refuse a request is useful, but it is not a replacement for those controls. --- Tools for Testing This Several tools can help automate or structure information-disclosure testing: - Garak: includes probes for sensitive information disclosure and prompt-leakage-related behaviors. - Promptfoo: supports adversarial testing and evaluations for LLM applications, including custom tests for sensitive-data exposure and prompt disclosure. Tools are useful for generating test cases and repeating requests, but they should complement application-specific testing. A model that reproduces a test value or reveals an instruction is not necessarily vulnerable; the tester still needs to establish what information was exposed, whether the disclosure was unauthorized, and what security boundary should have prevented it. --- What's Next in This Series This post covered three information-disclosure tests. Coming up: - AITG-APP-05 / 10 / 11 / 12 — unsafe outputs, harmful-content bias, hallucinations, and toxic output - AITG-APP-06 / 13 — agentic behavior limits and over-reliance on AI - AITG-APP-08 / 09 — embedding manipulation and model extraction - AITG-APP-14 — explainability and interpretability --- Official Resources - AITG-APP-03: Testing for Sensitive Data Leak - AITG-APP-04: Testing for Input Leakage - AITG-APP-07: Testing for Prompt Disclosure - OWASP Top 10 for LLM Applications — LLM02: Sensitive Information Disclosure - OWASP AI Testing Guide — the full AITG-APP category --- Final Thoughts The previous post examined whether attacker-controlled input can change an AI application's behavior. This post examines whether the application exposes information that should remain protected. Sensitive data leakage, input leakage, and prompt disclosure can produce similar-looking outputs, but they involve different information sources and different security boundaries. The important testing question is therefore broader than: > "Can I make the model reveal something?" A useful test asks: > "Can the application expose information to someone or somewhere that should not receive it?" Answering that question requires more than testing the model's response. It means tracing where information originates, how it enters the model's context, which tools and users can access it, and what happens to it after the model responds. Prompt design, output filtering, and model-level safeguards can all contribute to reducing disclosure. But effective protection ultimately depends on application-level access controls, appropriate data handling, session isolation, and a clear understanding of which information the model actually needs. Next in this series: unsafe outputs, hallucinations, and toxic content — testing what happens when the problem isn't information disclosure, but the content the application generates.

AI Security Testing 101: Prompt Injection and Indirect Prompt Injection This is the first post in a new series on AI application security testing, following the structure of the OWASP AI Testing Guide's AI Application Testing category (AITG-APP). Each post in the series will take one test from that category and turn it into something practical: what the vulnerability actually looks like, how testers approach it, and what "vulnerable" means in concrete terms. We're starting where the guide starts: prompt injection, one of the most well-known AI application vulnerabilities, and a class of attack that can underpin many other AI-specific security risks. --- Direct Prompt Injection (AITG-APP-01) Prompt injection happens when attacker-controlled input changes an AI application's behavior in a way that conflicts with its intended instructions or security constraints. The input may attempt to make the model ignore higher-priority instructions, reveal information it should not disclose, or perform an action it was not intended to perform. A useful, though imperfect, analogy is SQL injection: in both cases, attacker-controlled input can influence behavior that the application intended to control. The analogy has limits, however. SQL injection exploits the interpretation of data as executable syntax, while prompt injection arises from the way language models process natural-language instructions and contextual information. The AITG-APP-01 methodology characterizes a prompt injection in terms of three elements: - Instructions: what the attacker actually wants the model to do. - A trigger: the specific phrasing, framing, or encoding intended to cause the model to follow the attacker's instructions rather than its intended behavior. - Conflict with intent: the injected request conflicts with the application's intended instructions or constraints. Without that conflict, an unusual or adversarial-looking prompt may simply be a normal request rather than a security-relevant prompt injection. Prompt injection is also broader than jailbreaking. A jailbreak attempts to bypass safety restrictions, while prompt injection more generally involves attacker-controlled input changing model behavior in an unintended way. An injection can therefore be security-relevant even when it does not produce prohibited content. Whether an attack succeeds can depend heavily on how the trigger is constructed. That's one reason simple keyword filtering is insufficient on its own. Blocking the phrase "ignore previous instructions" does little against the same underlying idea expressed through role-play, another language, obfuscation, or an encoded representation. Attack Patterns Worth Knowing The OWASP guide catalogs a broad set of concrete payload techniques. They can be grouped into a handful of practical patterns: Persona and role-play framing Asking the model to "pretend" to be an unrestricted persona so that it treats otherwise disallowed behavior as being in character. A well-known example is the "DAN" jailbreak. The underlying technique is not limited to DAN. Attackers can construct fictional characters, simulated environments, alternate roles, or hypothetical scenarios intended to change how the model interprets its constraints. Context manipulation Telling the model to "forget" previous instructions, attempting to override higher-priority context, or gradually steering the conversation toward a prohibited behavior. This can include multi-turn approaches in which no individual request appears particularly harmful, but the sequence gradually changes the context presented to the model. "Crescendo" and "echo chamber" style attacks are examples of this broader pattern. Obfuscation and encoding Hiding or transforming the attack instruction using techniques such as base64, hexadecimal encoding, reversed text, unusual capitalization, deliberate typos, Unicode manipulation, or other transformations. The purpose is often to evade filters or exploit differences between what a security control recognizes and what the model can interpret. The important lesson is that filtering for a fixed set of phrases is not a sufficient defense against a language-based attack surface. Structural tricks Taking advantage of how the application or model handles structured inputs, including JSON or other formats, multi-message inputs, or instructions embedded within content that the application expects the model to process. Structured formats can create additional attack surfaces when applications treat model-generated or model-consumed fields as trusted simply because they conform to an expected schema. Contradiction and instruction-conflict framing Presenting multiple competing instructions or asking the model to produce both a normal response and an explicitly contrary response, with the goal of getting it to produce the latter. The exact payloads are less important than the general testing principle: don't assume that one known jailbreak or injection string represents the entire attack surface. These techniques are not permanent. Models, applications, and defenses evolve, so a payload that works against one model or configuration may become ineffective against another. That's why testing should combine techniques and repeat requests rather than relying on a single known payload. Response variability can also matter. In configurations where sampling introduces variability, the same prompt may produce different results across attempts. --- What Makes a Test Environment Different Before running tests against a real target, it's worth understanding what's actually in front of you. The architecture and model configuration can significantly affect which techniques are relevant and how the results should be interpreted. Temperature Lower temperature generally makes output more repeatable, while higher temperature generally increases variability and diversity. That matters during testing because a payload that succeeds once may not succeed consistently. Testers should therefore repeat relevant requests and record the configuration used rather than treating a single successful or unsuccessful attempt as definitive. Temperature does not by itself determine whether a system is secure, and a temperature of zero should not be treated as a universal guarantee of deterministic behavior. Fine-tuning vs. prompt tuning Safety and behavioral constraints can be implemented at different layers, including through model training or fine-tuning and through application-level instructions or other steering mechanisms. This distinction matters during testing because a successful bypass may target different layers of the system. A test might circumvent application-level instructions without demonstrating that the underlying model has changed, for example. The important question for a tester is therefore not just "did the model comply?" but also "which security control was bypassed?" Guardrails Guardrails can be implemented in several ways, including deterministic rules, classifiers, access controls, schema validation, filtering systems, or additional LLM-based checks. When an application uses an LLM as part of its guardrail system, that introduces another model and potentially another source of variability. Testers should evaluate the guardrail as part of the attack surface rather than assuming that adding another model automatically prevents prompt injection. Multi-agent setups When several agents hand work to each other, testers may only see the final output rather than the intermediate prompts, tool calls, and responses. That makes it harder to determine which agent or handoff an injection actually influenced. Testing therefore benefits from observability into agent-to-agent messages, tool calls, retrieved content, and intermediate decisions wherever possible. Tool access A model wired up to scripts, shells, databases, or external APIs can turn a successful injection from unwanted model output into an unauthorized action. Tool access can therefore substantially increase the potential impact of a prompt injection. However, tool access is not required for an injection to have security impact: unauthorized information disclosure or manipulation of important outputs can also be significant. --- Indirect Prompt Injection (AITG-APP-02) Direct injection assumes the attacker is the one providing the malicious input to the AI application. Indirect prompt injection works differently. The malicious instruction is planted somewhere the AI application will process as external content: a web page, PDF, email, uploaded document, database record, search result, or tool-related content. The model may then encounter that instruction as part of the context it receives. The important distinction is not that the model has literally "no way" to distinguish data from instructions. Rather, the model may fail to reliably distinguish untrusted external content from instructions when both are represented in the context it processes. That creates a different attack surface: the attacker may not need direct access to the AI application if they can influence content that the application will eventually retrieve or process. Indirect injection can be particularly concerning in applications that automatically retrieve external content, have access to sensitive information, or can take actions through tools. Concrete Surfaces This Shows Up On Hidden text in web pages or documents An attacker can place instructions in content that is difficult for a human reviewer to notice, such as HTML comments, invisible or visually obscured text, document metadata, or other content that is nevertheless extracted and passed to the model. The important security property is not that the text is literally invisible. It's that the application may process content that the user did not realize would be interpreted by the model. Structured data fields A JSON, XML, database record, API response, or other structured input may contain an attacker-controlled field whose value includes an instruction rather than the data the application expected. For example, a field intended to contain a document summary or description might contain language such as an instruction to ignore the application's task and perform another action. The fact that the content is valid JSON or XML does not make it trustworthy. Markdown-based exfiltration An indirect prompt injection can attempt to make the model include sensitive information in a URL or other externally observable location, including by generating a Markdown image reference or link containing attacker-controlled data. If the resulting content is rendered or fetched by a downstream component, this can create a data-exfiltration path. A related technique was demonstrated in the real-world EchoLeak vulnerability affecting Microsoft 365 Copilot. EchoLeak was a multi-stage attack involving indirect prompt injection and data exfiltration under particular conditions; it should not be treated as synonymous with every Markdown-based exfiltration technique. Encoded payloads External content can also contain encoded or otherwise obfuscated instructions. The same general filter-evasion issues that apply to direct prompt injection can therefore apply to indirect injection. Tool metadata and tool poisoning For AI systems using MCP or similar tool-calling protocols, tool names, descriptions, schemas, metadata, parameters, and outputs can become part of the model's context. In some client implementations, this information may not be prominently exposed to the user even though it is presented to the model. An attacker who can control a tool's metadata may therefore attempt to embed instructions into it, for example, instructions telling the model to retrieve information before using the tool. The security issue is not specific to one metadata field. The broader concern is that the model may treat untrusted tool-related content as instructions. The common thread is this: > Anywhere external content reaches the model without a reliable boundary between trusted instructions and untrusted data, there is a potential prompt-injection attack surface. That boundary is an architectural concern, not merely a question of finding the right payload string. --- What Counts as a Finding For both tests, a prompt injection becomes a security finding when attacker-controlled instructions cause behavior that violates the application's intended constraints or security boundaries. Examples include: - The model overrides or circumvents intended instructions in a security-relevant way. - The application discloses information that the model or user should not have access to, such as credentials, API keys, internal configuration, or another user's data. - The model or connected system performs an unauthorized or harmful action. - An indirect injection causes the application to treat attacker-controlled external content as an instruction and subsequently produce unauthorized disclosure or action. For indirect injection specifically, the distinguishing condition is that the security-relevant behavior originates in externally supplied content rather than the user's direct prompt. Simply following an instruction found in external content is not automatically a vulnerability. For example, if a webpage contains ordinary instructions that are relevant to the user's request and the model follows them as part of legitimately processing that page, there may be no security issue. The important question is whether attacker-controlled external content can change the application's behavior in a way that violates its intended security or operational constraints. --- Remediation, in Short Both tests point toward several overlapping defensive principles. Separate trusted instructions from untrusted content Clearly distinguish system/application instructions from user input and external data. Delimiters and structured context can help the model distinguish these sources, but they should be treated as mitigations rather than a guarantee that prompt injection will never succeed. Treat external content as untrusted Validate and sanitize external content before it reaches the model where appropriate. Consider hidden text, encoded instructions, unusual Unicode, metadata, and other forms of content that a human reviewer may not recognize as instructions. But don't rely on sanitization alone. Natural language is too flexible to make exhaustive filtering a reliable security boundary. Restrict model privileges Give the model only the access and capabilities it actually needs. If a model does not need access to credentials, private files, sensitive databases, or administrative functions, those capabilities should not be available to it merely because the model might request them. Authorization should be enforced outside the model wherever possible. Require approval for high-impact actions Sensitive, high-impact, or irreversible actions should require appropriate authorization or human approval rather than relying entirely on the model to decide whether an action is safe. The model should not be the sole security boundary protecting a privileged operation. Treat tool metadata as untrusted input Tool descriptions, schemas, metadata, parameters, and outputs should be treated as potentially untrusted content. Applications should avoid assuming that a tool definition is safe simply because it was supplied through an otherwise trusted tool-calling protocol. Add monitoring and observability Where practical, log and monitor retrieved content, model inputs, tool calls, authorization decisions, and significant outputs. This is particularly important in multi-agent systems, where a prompt injection may enter through one component and only become visible much later in the execution chain. --- Tools for Testing This Several tools can help automate or structure prompt-injection testing: - Garak: includes probes relevant to prompt injection and other LLM security testing. - Promptfoo: provides adversarial testing and evaluation capabilities for LLM applications, including prompt-injection testing. - Prompt Security Fuzz (ps-fuzz): a fuzzer focused on prompt-level attacks. Tools are useful for generating and repeating attacks, but they should complement application-specific testing. A generic payload that causes an unusual model response is not necessarily a vulnerability; the tester still needs to establish what security boundary was crossed and what the resulting impact is. --- What's Next in This Series This post covered the two prompt-manipulation tests. Coming up: - AITG-APP-03 / 04 / 07 — sensitive data leaks, input leakage, and system prompt disclosure - AITG-APP-05 / 10 / 11 / 12 — unsafe outputs, harmful-content bias, hallucinations, and toxic output - AITG-APP-06 / 13 — agentic behavior limits and over-reliance on AI - AITG-APP-08 / 09 — embedding manipulation and model extraction - AITG-APP-14 — explainability and interpretability --- Official Resources - AITG-APP-01: Testing for Prompt Injection - AITG-APP-02: Testing for Indirect Prompt Injection - OWASP Top 10 for LLM Applications — LLM01: Prompt Injection --- Final Thoughts Prompt injection isn't a vulnerability that can necessarily be "patched once." It is a recurring security problem for systems that allow language models to process instructions alongside user-controlled or externally sourced content. Direct injection tests what happens when a user actively attempts to change the application's behavior. Indirect injection tests whether attacker-controlled external content can influence the system even when the attacker never interacts with the AI interface directly. The important security question is therefore broader than: > "Can I make the model ignore its system prompt?" A useful test asks: > "Can attacker-controlled input or external content cause the application to violate an intended security or operational constraint?" That framing also makes the impact easier to assess. A successful jailbreak that produces harmless text is different from an injection that exposes confidential information, bypasses authorization, or causes a connected system to perform an unauthorized action. Prompt injection techniques will continue to evolve as models, applications, and defenses change. Effective testing therefore needs to combine different attack patterns, repeat tests under representative configurations, and most importantly evaluate the actual security boundaries around the model rather than treating the model itself as the only control.

Can AI Agents Reverse Engineer Your App? REA Deep Dive (MCP + Ghidra) Can a coding agent like Claude Code, Codex or Cursor take your own app apart? And can you believe what it reports? We open up REA (Reverse Engineer Anything), the open-source MCP server that lets AI agents drive local reverse-engineering tools like Ghidra, IDA and Hopper, and walk one app from first look to final report. ASH and MAY cover the whole system design: where REA sits between your agent and your tools, how every finding becomes an evidence record with a receipt-style ID, how it keeps observed, inferred and unknown apart, what it cannot see, the dual-use risk, and four practical layers to protect the apps you ship. What you'll learn: • What an MCP server for reverse engineering actually does, and what it doesn't • Why "coverage: unknown" and an empty result are not proof of absence • How REA picks one engine per target (no silent fallback) • Why this is not a sandbox, and what the job object really limits • Electron app defence: server-side decisions, ASAR integrity fuses, code signing, and obfuscation as a cost, not a wall CHAPTERS 0:00 Can an agent take our own app apart? 2:21 The map: where REA sits 11:58 The evidence: how we know it isn't made up 18:58 The investigation: first look to final report 26:50 Costs and risks: what it costs, where it breaks 32:15 Defence: if anyone can read it, what protects it 37:56 The verdict: can we trust it? LINKS AND SOURCES REA (Reverse Engineer Anything): morluto/rea REA README: github.com/morluto/rea/blob/main/README.md REA security boundary: github.com/morluto/rea/blob/main/SECURITY.md REA disclaimer: github.com/morluto/rea/blob/main/DISCLAIMER.md OWASP MASVS-RESILIENCE: mas.owasp.org OWASP MASTG, tampering and reverse engineering: github.com/OWASP/mastg/blob/master/Document/0x04c… OWASP Mobile Top 10, M7 insufficient binary protection: owasp.org Electron ASAR integrity: electronjs.org Electron fuses: electronjs.org Barak et al., On the (Im)possibility of Obfuscating Programs (2001): eprint.iacr.org Facts checked against REA at commit c7f6b8b (release 6.5.0) in October 2026. Kuna is on the main branch only. Things change fast, so check the repo for the current state. Responsible use: this video explains how the tool works and how to defend your own apps. It does not show how to get around licence checks, DRM or other protections, and neither does REA (see its disclaimer). Only analyse software you own or are allowed to analyse. ASH and MAY are animated characters. Their voices are AI-generated (Google Gemini text-to-speech). The invoice app is an illustrative example, not a real product. #ReverseEngineering #AIAgents #MCP