Adversarial ML and Red Teaming

About this paper

The capstone examination for practitioners who attack AI systems or defend against those who do. It assesses adversarial example theory, poisoning and extraction attacks, the evaluation discipline that separates real robustness from gradient masking, jailbreak methodology, and the scoping, measurement and reporting standards of professional AI red teaming. Expect questions that turn on subtle distinctions; partial knowledge will not pass.

What it covers

• Attack theory: adversarial examples, transferability, and white, grey and black box threat models
• Training-time attacks: data poisoning, backdoor triggers, and clean-label techniques
• Model confidentiality: extraction, membership inference, and the trade-offs of every defence
• Robustness evaluation: gradient masking, adaptive attacks, and certified versus empirical guarantees
• LLM offence: jailbreak taxonomy, automated attack generation, and multi-turn escalation
• Professional practice: scoping, rules of engagement, attack success measurement, and reproducible reporting

How it is marked

• Questions and answer options are shuffled for every sitting.
• Multi-answer questions are marked as a set: you need all of the correct options and none of the wrong ones. There is no partial credit.
• You need 80% to pass.
• You can revisit and change any answer until you submit.
• Afterwards you see every question, the answer you gave, whether it was right, and the reasoning behind it. The answer key itself is never printed, so the paper stays worth sitting.
• You can re-sit the paper, but not immediately: there is a short wait between attempts.