Friendly Fire Exploit Shows AI Coding Agents Can Be Tricked Into Running Malicious Code During Security Reviews
Published
Key Takeaways
- Friendly Fire exploit: AI coding agents executed malicious code during third-party security reviews via prompt injection attacks.
- Verified research: AI Now Institute demonstrated remote code execution against Claude Code and Codex default automation modes.
- User impact: Organizations should avoid granting AI agents unnecessary permissions when reviewing untrusted code repositories.
A newly published research brief from the AI Now Institute details a proof-of-concept (PoC) attack that can cause AI-powered coding assistants to execute malicious code while performing security reviews of third-party software. The researchers demonstrated the exploit against Anthropic's Claude Code CLI and OpenAI's Codex CLI under their default automated review settings, raising concerns about the security risks of using AI agents to analyze untrusted code repositories.
Researchers Demonstrate Remote Code Execution Through Prompt Injection
According to the AI Now Institute's official exploit brief, the attack targets AI coding agents configured to automatically review source code for security issues. The researchers tested the exploit against Claude Code CLI running Claude Sonnet 4.6, Claude Sonnet 5, and Claude Opus 4.8, as well as OpenAI's Codex CLI using GPT-5.5.
The proof-of-concept relies on prompt injection, a technique where hidden instructions are embedded inside files that an AI model reads. Instead of exploiting a traditional software vulnerability, the attack manipulates the AI agent into treating malicious content as legitimate instructions.
In the demonstrated scenario, an attacker modifies an open-source or third-party software library by inserting carefully crafted prompt injections into documentation files and adding files that appear to be part of routine security tooling. When a user asks Claude Code or Codex to perform a security assessment of the repository using the default automated modes - "auto-mode" for Claude Code or "auto-review" for Codex - the AI agent can be persuaded to execute a malicious binary without requesting additional user approval.
The researchers state that the attack works without requiring plugins, custom configuration files, Model Context Protocol (MCP) servers, hooks, or other additional integrations. Instead, it uses the tools' standard out-of-the-box configurations.
Attack Targets Security Workflows Using Third-Party Code
For the demonstration, the researchers modified a copy of the open-source Python library geopy with additional documentation and files containing hidden prompt injections. They stress that geopy was chosen only as an example and that the same approach could theoretically be adapted to other third-party or open-source repositories.
During the AI-driven security review, the embedded instructions lead the coding agent to believe that a bundled security script is necessary for completing the analysis. That script ultimately launches a malicious binary, resulting in remote code execution (RCE), a security issue that allows attackers to run code on the affected machine from a remote source.
The researchers also tested whether the AI models could detect the embedded prompt injections when directly asked to identify them. In their experiments, both Claude Sonnet 4.6 and GPT-5.5 failed to recognize the hidden instructions.
The AI Now Institute notes that results may vary because large language models are non-deterministic, meaning they do not always produce identical outputs. However, the researchers report that they successfully reproduced the attack multiple times across the tested model versions.
Why the Findings Matter for AI Security and Privacy Users
The report highlights risks for developers, security researchers, and organizations that rely on AI agents to automatically inspect software from external or untrusted sources.
Because these AI assistants can execute commands as part of legitimate security workflows, prompt injections hidden inside documentation or repository files may create new attack paths that traditional security tools are not designed to address.
The researchers recommend avoiding the use of AI agents to process untrusted data whenever those agents have permission to execute arbitrary code, access security-sensitive systems, interact with critical APIs, or feed automated pipelines that lack input validation. They also caution that relying solely on sandboxing or human oversight may not fully mitigate these risks, particularly in safety-critical environments.
The findings come directly from the AI Now Institute's official "Friendly Fire" exploit brief. The researchers also disclosed their findings to Anthropic and OpenAI, although they note that the proof-of-concept falls outside both companies' existing security disclosure policies. They further state that the attack may affect additional AI agents beyond those specifically tested because it exploits characteristics common to frontier AI systems rather than vendor-specific implementation details.
At this time, the Friendly Fire report brief does not recommend any immediate action for general VPN or privacy users. Instead, its recommendations are directed at organizations and security teams. The researchers advise against using AI agents to analyze untrusted data when those agents can execute arbitrary code or access security-critical systems, arguing that existing safeguards may not adequately mitigate the risks demonstrated in their proof of concept.
Related
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular