Technical Paper · G · Working Paper

AI Agent Scan: detecting AI agents inside a codebase

What an AI agent looks like in source code, the signals a scan reads for, the three ways the 2025 incidents hid a model inside a program, and one worked scan of a public repository that anyone can re-run.

Working paper · 24 September 2026 · figures from CodeDelta 2.1.1

Abstract

An AI agent inside a codebase is code that calls a language model and then acts on the answer: it may run a command the model wrote, edit a file, send a message, or call the model again. Such code is now ordinary in commercial repositories, and in 2025 it was also the shape of three public attacks. This paper describes how a static scan finds it. Four signal categories are read from every text file: AI SDK imports, agent orchestration, dynamic execution, and unchecked input reaching a prompt. One composition of those signals, a process launch or code evaluation within ten lines after a model call, is treated as the rogue pattern and rated CRITICAL without qualification, because it is the fingerprint the 2025 malware could not remove. The paper sets out the three coverage tiers that decide how deeply each file is read (full rules in 22 languages, endpoint-and-launch rules for scripts and CI, a text-level pass for everything else), then walks a scan of the public demonstration repository, file by file, whose 74 fully-covered files yield 29 CRITICAL, 2 HIGH, 27 ELEVATED and 16 NORMAL results. The figures are frozen in the product's test suite and the repository is public, so the scan can be repeated by anyone. The paper closes with what a static scan cannot promise. Companion papers B and F survey the wider method classes; this one describes a working instrument.

01What an agent inside a codebase is

The word agent is used loosely. For the purpose of a source scan it has a narrow meaning: code that sends text to a language model and then does something with the reply other than show it to a person. The doing is what matters. A chat window that prints the answer is an AI feature. A function that takes the answer, treats it as a shell command and runs it is an agent, and it is an agent whether it was written by a developer on purpose, generated by an assistant, or planted by an attacker.

Three questions therefore decide whether a file matters to the scan, and in what degree:

The first question finds AI use, which is common and mostly benign. The third finds the thing a reviewer needs to see first. The scan's ratings are built on that order.

02The four signals and the rogue pattern

Every file the scan opens is read for four categories of signal, in the scanner's own terms:

SignalWhat is matchedWhat it establishes
AI SDK importsPer-language import tables for the AI client libraries, plus the 34 model-host names the scanner knows (hosted providers, enterprise cloud routes, and local servers) matched in the source text.The file is an AI integration point.
Agent orchestrationCode that launches or controls autonomous agents: agent frameworks, and the command-line agents (claude, gemini, codex, copilot, aider and others) invoked with a prompt, with the permission-bypass flags named when present.The file starts an agent.
Dynamic executionProcess launches, shells and code evaluation, from a per-language table, with comments and string literals stripped first so that prose about a command is not a command.The file can act.
Prompt injection vectorsUnchecked input (a request field, a file, an argument) flowing into a prompt.The model can be steered from outside.

Each signal adds to a per-file score from 0 to 100. The score sets the rating: ELEVATED from 40, meaning the file uses AI and deserves a look; HIGH from 70, or whenever an SDK and an execution construct are both present, meaning the ingredients of an agent are in one file.

Above the score sits one rule that does not depend on it. If a process launch or a code evaluation occurs within ten lines after a model call, the file is rated CRITICAL, unconditionally. This is the rogue pattern: the model's output is in a position to be executed. It is the one signal that cannot be designed out of LLM-enabled malware, because removing it disables the malware; it can only be obscured, and Section 06 is honest about how. The ten-line window is deliberate. A proximity rule can be explained, checked by eye in the report, and tested with fixtures whose expected result is written before the scanner runs. A rule that tried to follow data across a whole program could not be.

Two further rules feed the same report. A model called inside a loop is flagged for cost and runaway risk. And the tree as a whole, not only its source files, is checked for agent infrastructure left behind: agent configuration directories, gateway tokens, and committed credentials in key formats, which are reported with their values redacted.

03The three attack shapes of 2025

Three public incidents in 2025 each hid a model inside a program in a different way. The scanner's rules were built to name each shape, and the product's demonstration repository carries an inert sample of each so that the output can be seen without running anything dangerous.

A hijacked agent CLI. In August 2025 a compromised release of the nx build tool shipped a post-install script that ran the developer's own Claude, Gemini and Q command-line agents with their safety checks switched off, and used them to search the machine for wallets and credentials [1]. The model never appears in the source as an API call; it is reached through a tool already installed on the victim's machine. The scan reports the agent CLI by name, the line it is launched on, and the bypass flag.

An encoded prompt. In July 2025 Ukraine's CERT reported LameHug, malware that kept its instructions to the model as a base64 string, sent them to a hosted model, and ran the commands that came back [2]. A scan that reads only plain text sees a harmless-looking literal. The scanner therefore makes a second pass over base64 and hex literals, compressed or not, decodes them once, and reads the result as text: an instruction to a model inside an encoded string is reported with its opening words quoted, and a model host assembled from an encoded string is reported as a hidden host.

A local model, no address in the file. In August 2025 ESET described PromptLock, ransomware that generated its scripts at run time from a locally hosted open-weight model [3]. A program talking to Ollama need never write a URL: it reads an environment variable or calls the client library. The scanner reports the local endpoint from the library call or the variable alone, and, where the reply is launched, the rogue pattern.

Each shape defeated a naive check in a different place: the first has no model call in the code, the second has no readable prompt, the third has no address. That is why the scanner reads for launches, decodes literals, and knows the local servers, rather than matching on a list of API hostnames alone.

04Coverage: which files are read, and how deeply

A scan that silently skips files it does not understand is worse than one that says so. Every text file in the tree is read, and every result records the level at which its file was read, so a clean result can never be mistaken for a file that was not examined.

TierFilesRules appliedCeiling
Full22 languages: C, C#, C++, Dart, Elixir, Go, Groovy, Java, JavaScript, Julia, Kotlin, Lua, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Swift, TypeScript. Web pages and notebooks are opened and their script blocks and code cells scanned as the language they hold.Comment and string stripping, SDK table, model-call idioms, launch and evaluation table, loop rule, rogue rule.CRITICAL
EndpointShell scripts, CI recipes, Dockerfiles, and languages with no AI SDK ecosystem.Comment stripping, model hosts, launch table, rogue rule. The curl model | bash shape lives here.CRITICAL
TextEvery other readable text file under a size cap.Model hosts, the encoded-literal pass, credentials.ELEVATED (no launch table, so the rogue rule cannot fire)

The tier is a property of the file type, recorded on each result and printed in the report, the JSON and the CSV. Coverage is also tested as a property of the product: the test suite fails if a file extension the scan should recognise is dropped, and if a language the scan applies rules to is one the churn engine cannot parse.

05A worked scan: the public demonstration repository

The product ships a demonstration project, fetched on demand from a public repository, so that a first scan has something to find. It is a small fictional help-desk system with AI features wired in the way a real one would have them, plus inert files written in the shape of each 2025 incident. Its expected results are frozen in the product's test suite and re-checked on every build, so the figures below are not a description of one run but the scanner's committed behaviour on that tree.

The 74 fully-covered files span 27 file types. The scan rates them:

RatingFilesWhat it means in this tree
CRITICAL29A model call whose reply is in a position to be executed. Among them: a post-install script that launches an agent CLI; three beacons whose instructions are encoded (plain base64, hex, and compressed); an assistant that talks to a local model with no address in the file; an auto-fix routine that runs what the model returns; a notebook whose code cell does the same; a web page whose script block does the same.
HIGH2An SDK and an execution construct in one file, not within the rogue window: a CrewAI orchestrator, and a digest job that calls a China-hosted model inside a loop.
ELEVATED27Ordinary AI use: a classifier, a sentiment model, a summariser, chat assistants in several languages, and provider modules that send data to models hosted abroad.
NORMAL16No AI use: the ticket store, date utilities, MIME tables, status codes, tests, logo data.

The same tree, scanned by the GitHub Action on a pull request against the public repository, produced the summary that anyone can read on that pull request today [4]: 64 CRITICAL · 7 HIGH · 81 ELEVATED across the whole repository (which holds a second, older sample set alongside the help-desk), 17 files at endpoint level and 57 at text level, 14 files sending data to foreign-hosted models, 2 files calling a model inside a loop, 16 agent-infrastructure artifacts of which 4 are rogue residue or credentials, and 2 committed keys in recognised formats, values redacted. The run took 23 seconds on GitHub's standard runner.

Three things about that summary are the point of the worked example. First, the CRITICAL count is dominated by inert samples written to trip the rule, which is what a demonstration is for; on a repository that uses AI in the ordinary way the CRITICAL line is usually zero and the ELEVATED line is where the information is. Second, the endpoint-level and text-level counts are not padding: they say how many files were read at a lower tier, so a reader knows what a zero means for them. Third, the credentials and the agent residue were found in the tree, not in the source, because an agent leaves things behind that a source-only scan would walk past.

06What a static scan cannot say

Findings are patterns in source that warrant a person's look, not confirmed malicious activity, and the report says so at the top of every run. The limits are specific:

Against careless concealment, the encoded prompt, the environment-variable host, the agent run from a CI step, this is a first tier of detection that did not exist before, and it costs one pass over the tree.

07Running it

The scan runs in three places, on the same engine. On GitHub, one workflow file runs it on every pull request and posts the summary as a comment; the pull request cited above is the live example. On the desktop, the Agent Scan mode produces a report with one card per file, the rating, the reasons and the cited lines, and a map of every file that talks to a model. From the command line, codedelta-gui scan <dir> --mode agent writes the same report, the JSON and the CSV, with the coverage tier on every row.

Reproducing the figures. The demonstration repository is public at github.com/code-delta-app/demo. Open pull request #4 to read the Action's summary, or add the workflow from the GitHub quick start to a fork and run it yourself. The desktop app fetches the same tree with one press from its Try It card.

·References

  1. nx (2025). Security advisory GHSA-cxm3-wv7p-598c: compromised nx releases with a post-install script invoking installed AI command-line agents. github.com/nrwl/nx/security/advisories/GHSA-cxm3-wv7p-598c
  2. Cato Networks, Cato CTRL Threat Research (2025). Analyzing LAMEHUG, on the CERT-UA report of malware whose commands were generated at run time by a hosted model from a base64-encoded prompt. catonetworks.com/blog/cato-ctrl-threat-research-analyzing-lamehug/
  3. ESET Research (2025). First known AI-powered ransomware uncovered (PromptLock), scripts generated at run time from a locally hosted model. welivesecurity.com/en/ransomware/first-known-ai-powered-ransomware-uncovered-eset-research/
  4. CodeDelta demonstration repository, pull request #4: the Action's summary on the public tree. github.com/code-delta-app/demo/pull/4
  5. CodeDelta (2026). Detection of AI-Generated Code and Embedded AI Agents in Source Projects: A Survey of Method Classes. Technical Paper B. paper-ai.html
  6. CodeDelta (2026). Detecting LLM-Enabled Malware and Insider-Planted Backdoors in Source Repositories: A Survey of Signals and Their Limits. Technical Paper F. paper-agent-malware.html
  7. The Agent Scan product page, with the scanner's real output on the three inert incident samples. codedelta.app/agent/