Secure AI Application Engineering
Proves you can design, build and ship LLM-backed systems that stay safe under adversarial pressure.
An advanced examination for engineers and architects who build applications around large language models. It assesses threat modelling of LLM systems, trust boundary design, sandboxing and egress control, agent tool architecture, model supply chain integrity, and safe release engineering. Candidates should have hands-on experience shipping at least one production LLM feature.
- Threat modelling LLM applications: trust boundaries, untrusted model output, and injection paths
- Agent and tool security: least privilege per tool, capability tokens, and schema validation
- Runtime containment: code interpreter sandboxing, egress control, and SSRF defence
- Economic and availability attacks: denial of wallet, rate limiting, and quota design
- Model supply chain: unsafe deserialisation, registry integrity, signing, and plugin risk
- Release engineering: caching hazards, secret handling, evaluation gates, and safe rollback
Learn the material
8 modules, about 50 minutes of reading. Work through them in order, or jump to whatever you need. The assessment is drawn from exactly this material.
- 01 Threat modelling an LLM application: where the boundaries actually sit Why the model belongs outside the trusted zone on both its input and its output side, and how to enumerate every path attacker-controlled text takes into context. 7 min
- 02 Tool design, least privilege and capability tokens Why the privilege an agent holds lives in its tools rather than in its instructions, and how scoped, short-lived credentials bound the damage when the agent is steered. 6 min
- 03 Treating model output as untrusted: schema validation and encoding at the sink Why well-formed structured output is not safe output, and why sanitisation has to happen at each destination rather than once at generation. 6 min
- 04 Containment: sandboxes, egress control and server-side request forgery The controls that actually bound the damage when model-generated code or model-chosen URLs do something hostile, and why screening the code first is not one of them. 6 min
- 05 Denial of wallet: cost-weighted limits and quota design Why an economic attack needs an economic control, and how to build quotas that survive agentic fan-out. 6 min
- 06 The model and plugin supply chain Why loading a model file can be code execution, what a co-hosted checksum really proves, and how a tool description becomes an injection vector. 7 min
- 07 Secrets and shared infrastructure: prompt caches, logs and system prompts Why a system prompt is not a vault, and how a cost optimisation on a shared inference path becomes a cross-tenant disclosure. 6 min
- 08 Shipping safely: evaluation gates, atomic versioning and rollback How to catch behavioural regressions that unit tests cannot see, and why reverting the code does not revert the behaviour. 6 min
Take the assessment
19 questions drawn from the material above. Pass and you can put your name to a certificate with a serial anyone can verify.
How it is marked
- Questions and answer options are shuffled for every sitting.
- Multi-answer questions are marked as a set: you need all of the correct options and none of the wrong ones. There is no partial credit.
- You need 75% to pass.
- You can revisit and change any answer until you submit.
- Afterwards you see every question, the answer you gave, and whether it was right. The answer key is never printed.
- The reasoning behind each answer is released once you pass. Held back on a fail, it would hand over most of the key to anyone willing to sit the paper once and read it, which is why the taught material above is the intended route back.
- You can re-sit the paper, but not immediately: there is a ten minute wait between attempts on the same course.