The model and plugin supply chain

A data scientist wants to load a community-published checkpoint from a public model hub onto a shared cluster. Ask where the risk begins and most people say inference: a backdoored model that misbehaves on trigger inputs. That risk is real but it is not the primary risk in the act of loading. Legacy checkpoint formats are built on a general-purpose object serialisation format that reconstructs arbitrary objects by executing code during deserialisation. Loading the file is the code execution event. It happens before a single inference runs, with the privileges of whoever ran the load, on a machine that usually has cluster credentials and network access.

This is why weights-only serialisation formats exist: they store tensors and metadata with no executable payload and no object graph to reconstruct, which removes the class rather than mitigating it. The practical rule is to prefer weights-only formats and refuse pickle-based checkpoints from external sources. Be clear about what this buys: it eliminates load-time code execution. It does not tell you anything about backdoors in the weights themselves, which is a behavioural property and needs behavioural evaluation. The other plausible-sounding concerns, an oversized checkpoint exhausting cluster memory or a licence prohibiting commercial use, are real operational and legal issues but they are not the security property of the load.

Integrity: what a checksum proves and what it does not

A registry publishes checksums for its model artefacts on the same web server that hosts the artefacts, and deploy pipelines verify the checksum before rollout. This is genuinely useful: it detects corruption in transfer and accidental substitution. It is not an integrity control against an adversary, because whoever can tamper with the artefact can rewrite the checksum in the same breach, and your pipeline will happily verify the attacker's file against the attacker's digest and report success.

What provenance requires is a signature under a key the hosting server does not hold, verified at deploy time against a pinned set of trusted identities. The trust then rests on key custody rather than on server custody, and compromise of the distribution host no longer implies compromise of the artefact. Note the wrong diagnoses that circulate here: the hash algorithm is not the problem and swapping it for another does not help; moving verification to training time rather than deploy time verifies a different moment and leaves the deployment unprotected; and adding an antivirus scan does not establish provenance.

That last point is worth stating plainly because it appears in review checklists. Conventional antivirus engines match signatures of known executable malware. They have essentially no coverage of tampered tensor values and limited coverage of embedded serialisation payload variants, so scanning weight files is not a tamper check and must not be relied on as the primary one. Equally, mirroring external artefacts into an internal store is good hygiene for availability and for pinning, but it does not launder trust: if you treat everything in the mirror as trusted thereafter, you have simply moved the unverified artefact closer to production.

1. Fetch into quarantineDownload to an isolated location with no cluster credentials and no ability to load the artefact. Nothing in this step executes the file.
2. Verify a signature, not just a digestCheck the publisher signature against a set of trusted keys pinned locally in your pipeline. A digest published beside the artefact only proves the file matches what that server says it should be.
3. Pin the exact digest in deployment configRecord the artefact digest so a re-tagged upstream file cannot silently substitute later. Tags move; digests do not.
4. Require a weights-only formatReject pickle-based checkpoints from external sources. If a conversion is unavoidable, do it inside a disposable isolated environment that holds no credentials.
5. Load first in an isolated environmentEven for weights-only artefacts, first load in an environment with deny-by-default egress and no production identity, so an unexpected loader path cannot reach anything.
6. Evaluate behaviour before promotionBackdoors live in the weights, not the container. Run your safety and adversarial suites against the candidate before it is eligible for production.
7. Mirror internally, but keep verification enforcedAn internal mirror helps availability and pinning. It does not confer trust: verification must happen on the way in and stay enforced on the way out.

Plugins: the description is part of the attack surface

A platform lets third-party developers publish tool plugins for your agent, each with a natural-language description the model reads when deciding which tool to use. Your review process reads the plugin's code. It should also read the description, because that text is injected into the model's context on every request, which makes it an instruction channel controlled by the plugin author. A hostile description can bias tool selection towards itself, tell the model to pass along additional context, or embed instructions that steer the agent's behaviour generally. There is no malicious code anywhere in this attack, which is precisely why a code-only review misses it.

Treat descriptions as untrusted content: review them as text, pin the version so an approved plugin cannot silently change its description after review, require re-approval when the metadata changes, constrain length and formatting, and isolate each plugin's credentials so a plugin that does win the model's attention still cannot reach anything outside its own scope. The other concerns raised about descriptions are mostly not security issues: long descriptions do add token cost, and inaccurate ones do degrade tool-selection quality, but the injection path is the one that changes what the agent does on an attacker's behalf.