AI safety · The AI Threat Index

We pointed five threat instruments at the 100 most-relied-upon packages less than two years old. Here is everything they found — and everything they didn’t.

Young packages with hundreds of millions of downloads are the least-audited code your build pulls in. So we audited them — deterministically, in public, with instruments anyone can re-run. The same scan runs on your code with Code Delta.

1 September 2026 · the index itself: github.com/code-delta-app/ai-threat-index · in this series: the Hugging Face swarm · the safety test that deleted the machine · the portfolio: the hidden dangers in your code

The index!

The public AI Threat Index repository on GitHub: the method-is-the-workflow section, the disclosure policy, the 13 individually-inspected credential matches, and the results table with a reviewed-benign-fixtures label

Not an artist’s impression — the index itself, live at github.com/code-delta-app/ai-threat-index. Three things to notice: the method section links the workflow that produced every number; the line above the table says all thirteen credential matches were individually inspected; and the AWS row’s credential cell reads “reviewed: benign fixtures” — the disclosure policy running in code, not prose. Click the image to open the repository.

Nobody picks the packages

The credibility of a scan like this dies the moment someone chooses the targets. So nobody does. A selection script in the public repository derives the cohort from registry metadata alone: every npm and PyPI package whose first release was within the last 24 months, ranked by lifetime downloads, top fifty per registry. Change nothing and you get the same hundred names we did. The scan itself runs as a public GitHub Actions workflow on GitHub’s own runners, downloading the same CodeDelta engine anyone can download, and commits its results back to the repository where the run logs are public. The method is the workflow. There is no private pipeline to trust.

Why packages under two years old? Because the ten-year-old head of the dependency curve — the lodashes and requests of the world — is the most-audited code on earth, and when we scanned that tier it came back near-silent. The young tier is the opposite: names that did not exist two Christmases ago now carry hundreds of millions of downloads, maintained by small teams, adopted faster than anyone could plausibly review them. That is where the incident record says supply-chain trouble actually lives.

Finding one: the AI agent stack is enormous exposed surface

The young npm tier is mostly micro-packages — utilities with 600-million-download reach and a dozen files — and showed essentially no AI surface. The young PyPI tier is the AI agent stack, and the numbers deserve staring at. fastmcp: 699 of its 846 files carry agent-scan signals. claude-agent-sdk: 67 of 88, most rated HIGH — model output adjacent to execution, which is an agent framework’s entire job description. strands-agents: 354 of 628. google-adk: 329 of 787. pydantic-ai-slim can reach fifteen distinct model providers from one install; mlflow-tracing integrates six providers and ten agent frameworks. None of this is an accusation — it is what these packages are for. It is also, the moment one lands in your dependency tree, your codebase’s new ability to talk to companies you never chose, in jurisdictions you never reviewed. The index makes that surface visible per package, before you install it.

Finding two: thirteen credential alarms — and what looking found

The credentials instrument matches documented vendor key formats — no scoring, no guesswork — and across the cohort it fired thirteen times. We inspected every single one before writing this sentence, and all thirteen are deliberate, benign fixtures. AWS’s own SDK carries AWS’s own documented example key in its type annotations. Google’s GenAI test suite holds GitHub tokens of the form ghp_xxxx… — placeholders, x’d out. Two projects keep private-key headers in JWT test fixtures and documentation examples with no key material attached. And the best of the lot: prek, a fast reimplementation of pre-commit, was flagged seven times because its own private-key detector carries the patterns it hunts. Our scanner found their scanner, and both were right.

Two lessons ride on that. First, the format table works: every alarm was a real key-format match at a real file and line, checkable in seconds. Second, this is precisely why CodeDelta ships the credentials merge-gate off by default: the ecosystem’s most-relied-upon young packages are strewn with deliberate inert fixtures, and a gate that broke their builds uninvited would be a nuisance, not a defence. Detection always-on, enforcement opt-in — the field data says that default is right.

Finding three: one build hook, zero fetchers

The xz-class questions — does anything run code on install, does anything download remote content at build time — came back about as quiet as they can. Zero remote fetchers across the cohort. One install-hook flag: librt, the mypyc runtime library, whose setup.py defines a custom build step. We read it. It compiles C extensions and, in a test-only branch, builds a copy of Google Test that is vendored into the repository, with a local make — no network anywhere. Maintained by the mypy core team. A true positive for the instrument, a clean bill for the package.

What this clean bill does not mean

A finding is only as honest as its limits, so here are ours. A snapshot is not a warranty. These packages were clean on the day of the scan; the supply-chain incidents that matter are changes — the maintainer swap, the version that fetches from somewhere new — which is why the index re-runs monthly and the interesting output over time will be the diff, not the table. Nine packages could not be audited at all: NVIDIA’s CUDA wheels publish no source — binary blobs, trusted on signature alone, and no scanner on earth can read what was never published. The npm rows under-scan: published npm packages ship their code inside dist/ folders, which the current engine deliberately skips as generated output — the right default for scanning a development repository, a blind spot for published artifacts, and a disclosed one we are addressing. And throughout: flagged files are pointers for review, never verdicts — an agent framework full of HIGH flags is doing its declared job, and the value is knowing the surface exists, not pretending every flag is a wound.

Watch it with us — or point it at your own code

The index re-runs on the first of every month, in public, and the repository history is the record. If a provider list quietly grows, a fetcher appears in a build file, or a credential lands somewhere it shouldn’t, the diff will say so — deterministically, with no model in the loop deciding whether to raise the alarm. The same five instruments run on your own repositories today: the threat-detection portfolio shows each layer with real output, the mini guide covers running and gating them, and Code Delta is free to evaluate — nothing leaves your network, which is more than can be said for your dependency tree.

Sources

  1. The AI Threat Index — repository: cohort rule, workflow, full results and run logs
  2. The engine the workflow downloads — public releases (v1.9.9 at time of writing)
  3. AWS’s documented example access key — AWS IAM documentation (the AKIAIOSFODNN7EXAMPLE family)
  4. librt — PyPI: the mypyc runtime library; the inspected setup.py ships in its source distribution
  5. prek — PyPI; the detector whose patterns our detector detected