From Coding Mistakes to Misplaced Trust: How AI Tools and Dependencies Expand the Attack Surface
Published
Key Takeaways
- During incidents, shared infrastructure, account activity, and metadata often reveal links that the payload may not.
- The payload itself rarely connects cases, and my advice to analysts remains to look beyond the code.
- AI agent tooling combines supply chain risks with privileged access, making it one of the riskiest dependencies.
- Sharma’s recent research shows that continuous monitoring of agent behavior reveals more than a one-time review at adoption.
- Manifold Security warns that traditional scanners focus on code, but AI tool metadata can quietly shape an agent's behavior.
In this Expert Insights, Ax Sharma, Head of Research at Manifold Security, explains how human errors, misplaced trust, and overlooked metadata impact software supply chain attacks.
Drawing on his work at Sonatype and BleepingComputer, Sharma brings years of experience investigating software supply chain attacks, malicious open-source packages, and emerging AI security risks.
He looks beyond malicious code to the account activity, infrastructure reuse, repository mismatches, and network behavior that can connect separate incidents.
The discussion shows why traditional incident response methods may miss threats that appear only during an AI agent’s runtime. Open source packages, CI/CD systems, and trusted platforms such as GitHub can all become pathways for access when permissions are poorly controlled.
Sharma offers practical signals on how to investigate compromised components, suspicious developer tooling, and attacks that hide in systems that are considered trustworthy.
Vishwa: Looking at the incidents you've investigated, what attack paths or intrusion patterns point to human errors?
Ax: Software supply chain attacks I’ve tracked and reported on in the last decade have always been built on small human errors. Traditionally, a developer might misspell a dependency name and accidentally fetch a typosquatted artifact.
Dependency confusion made the problem worse: the exploit is contingent on nothing more than an attacker squatting the name of your internal dependency, one you'd left exposed in public source code or build manifests, to push their payload into your environment.
The developer never mistypes anything; the error happened upstream, and the build system does the rest automatically. In the AI era, human error remains but has evolved.
- A developer asking an LLM to fetch a PyTorch dependency is trusting the model to find and pull the right artifact, not hallucinate a plausible-sounding package name or grab an unofficial, counterfeit component that could turn out to be malicious or something more subtle.
- A simple SKILL.md file, for example, could tell your agent to deliberately write broken code, produce broken "fixes" in a loop, and actively hide the fact that the skill exists – all in plain English.
The mistake has moved from the keyboard to the delegation itself: we've swapped a typo for misplaced trust in an intermediary.
Vishwa: When findings from separate cases are connected, what links between attack vectors, infrastructure, or tradecraft do you look into?
Ax: The payload itself rarely connects cases, and my advice to analysts remains to look beyond the code. The metadata around a sample is usually a stronger indicator:
- Author email reuse
- Account registration timing
- Shared exfiltration endpoints (Discord webhooks, paste sites, Telegram bots), and
- Naming conventions across packages.
Campaigns tend to spread across dozens of packages/components and accounts, but operators get lazy about infrastructure. In a recent investigation into new AI asset registries like ClawHub, for example, we linked 23 misleading plugins across 15 publisher accounts through registration patterns and use of a common scope.
Code reuse is another thread, though it needs handling with care. Attackers sometimes copy their own obfuscation and helper functions between campaigns, so a distinctive string-decoding routine or an odd variable naming habit can tie together packages published months apart under different names.
But code also gets copied by copycats, lifted from public write-ups and open malware repos, so on its own it's a lead, not an attribution. It carries far more weight when it converges with the infrastructure and account-level signals above.
Vishwa: Which stages of the intrusion lifecycle have produced the most useful observations in your research?
Ax: In traditional supply chain incidents, initial access and early execution were the goldmine. Install-time hooks and first-run beaconing told you almost everything about a malicious package within minutes, because the payload was front-loaded by design.
AI agents break that model. An agent can run cleanly through several sessions and then act unpredictably on an arbitrary tool call, sometimes because a skill is buggy or an employee configured it carelessly, and sometimes because an attacker hijacked it mid-session through indirect prompt injection or instructions planted in an untrusted webpage the agent fetched.
There is no single revealing moment to inspect, which is why the most useful observations in my recent research come from watching agent behavior continuously rather than examining the artifact once at the point of adoption.
Vishwa: What details or indicators show up repeatedly but receive little attention?
Ax: Metadata and account-level events. Repository mismatch is a big one. A package, extension, or AI plugin links to a legitimate GitHub repo that does not actually contain the published code, so anyone doing due diligence reviews the wrong source while the package borrows the repo's credibility.
A new maintainer suddenly added to a big project can be an early sign of a hijack in progress. Social engineering keeps working too, whether it's compromising a developer account to publish trojanized artifacts, or simply taking over an expired maintainer domain to resurrect an abandoned but popular open source project, this time shipping malicious versions.
On the network side, watch egress traffic not just to suspicious hosts but to legitimate ones. GitHub and Pastebin get abused as command-and-control (C2) precisely because enterprise monitoring waves them through.
With AI registries, the same problems are recurring, but the attack surface has become far more unpredictable due to the dynamic nature of runtime.
The under-examined layer there is tool descriptions and manifests: instructions for the model can be embedded in metadata that no traditional scanner may ever read, because it is not code.
Vishwa: What dependencies do attackers keep exploiting to move deeper into an environment?
Ax: The most durable targets are the dependencies that survive security review because the business can't function without them.
Examples would include the network monitoring tool with privileged access to every endpoint, the CI/CD pipeline holding long-lived secrets, the service account that authenticates everywhere and expires never. Compromising any one of them converts a foothold into reach across the environment.
AI agent tooling is the newest addition to that list, and the riskiest, because MCP servers and agent skills arrive through supply-chain-style channels but land with the privileges of a management agent, shell execution and credential access included, adopted with the casualness of installing a library.
Vishwa: How are attackers combining multiple techniques?
Ax: By stacking individually trivial tricks into a credibility chain: a lookalike name, inflated download counts, a copied README, all of which may be enough to make a counterfeit AI plugin or skill look legitimate to someone glancing casually.
Delivery then leans on channels defenders already trust: GitHub, Pastebin, or a SKILL.md that
reads benign but fetches code from an external host serving non-deterministic content.
Meaning, if you check the URL in a browser, it returns clean code. But should it detect an IDE or AI agent via the ‘User-Agent’, it quietly switches to serving the actual payload. A scanner
passes the URL as clean while it mutates in a real dev or prod environment.
Combine that with an over-permissioned agent and an open egress channel, and a single planted instruction becomes data theft or code execution. Every ingredient looks benign in isolation, which is exactly how the combination evades point-in-time review.
Vishwa: What attack surfaces keep appearing regardless of the victim sector?
Ax: The human, first and always. Targeted phishing and social engineering for credential theft work in every sector because the weakest link doesn't vary by industry. Beyond that, the technical surfaces are converging: every sector runs on the same open source registries, the same CI systems, the same IDEs, and increasingly the same agent frameworks, so one malicious package or poisoned agent tool can land in a bank, a hospital, and a game studio in the same afternoon.
The two surfaces are merging, too. An AI agent acting on planted instructions is social engineering with the human removed from the loop. Sector matters less every year; what you run, and what it does while running, matters more.
Related
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular
Get expert insights on threats, breaches, scams, and security trends — delivered every Monday.
Most Popular