Module 3 · 6 min read
Confidentiality, Integrity and Availability for Models and Data
How the classic triad maps onto model weights, training data, prompts, outputs and serving capacity, and the classification traps that catch people.
The CIA triad is not superseded by AI. It works perfectly well, provided you are disciplined about naming the asset before you name the property. AI systems have more distinct assets than most services, and the same attack can look like a different property depending on which asset you have in view. Start by listing them explicitly: the model weights, the training and fine-tuning data, the prompts and context assembled at runtime, the generated outputs, the serving infrastructure and its capacity, and the credentials held by the tool layer.
| Property | Training-time example | Inference-time example |
|---|---|---|
| Confidentiality | Sensitive records absorbed into weights and later recoverable | System-prompt leakage, model extraction, cross-tenant exposure |
| Integrity | Poisoned data or an implanted backdoor trigger | Injected instructions redirecting behaviour; tampered retrieval corpus |
| Availability | Corrupted or lost training artefacts and pipelines | Resource-exhausting inputs, quota exhaustion, cost amplification |
Confidentiality
Consider an attacker who repeatedly queries a proprietary model through its public API and uses the responses to train a close functional copy. Nothing has been altered. Service was not denied; in fact the attacker paid for every call. What has happened is that a confidential asset, the model's behaviour and by extension the investment embodied in its weights, has been disclosed and replicated without authorisation. That is a confidentiality violation, and it is the standard framing for model extraction.
Try it first
An attacker recovers your full system prompt, including the list of tools the assistant can call and the internal terminology it uses. Nothing else leaks. Before reading on: which CIA property is affected, and why is this more than an embarrassment?
It is a confidentiality violation: operator-authored content was disclosed without authorisation, and nothing had to change for the harm to occur. It matters beyond embarrassment because the system prompt is reconnaissance. It names the mounted tools, which tells an attacker exactly which capabilities to aim an injection at; it reveals internal terminology, which helps craft content that will win retrieval; and it exposes whatever defensive wording you rely on, letting the attacker rehearse against it. In poorly built systems it also exposes credentials, which turns a disclosure into an immediate compromise.
Confidentiality in AI systems has several other faces. Training-data extraction and memorisation expose the corpus through the model. Membership inference exposes whether a specific record was used. System-prompt extraction exposes the operator's instructions, tool inventory and sometimes, when teams have been careless, credentials. Retrieval layers leak when an index is built without per-user access filtering, so a query returns a chunk the asker was never entitled to read. And in multi-tenant deployments, isolation failures let one customer's data surface in another's session.
Integrity
Integrity splits into integrity of the model and integrity of its behaviour in a given session. The first is attacked at training time by poisoning, and in the supply chain by tampering with a distributed artefact. The second is attacked at inference time: injected instructions do not change a single weight, but they change what the system does, which is what the business cares about. Retrieval corpora are an underrated integrity asset in their own right. If an attacker can edit the wiki, the ticket archive or whichever store your retriever reads, they control part of the model's evidence base, and a confidently wrong answer sourced from a tampered document is an integrity failure even if no tool is ever called.
Availability
Availability includes everything familiar from ordinary services, plus one dimension that is sharper here. Inference has a variable and substantial per-request cost, and much of it is metered. A user who submits inputs deliberately crafted to maximise the compute the model spends on each request inflates latency for every other user and inflates the bill at the same time. The service may never fall over; it simply becomes slow and expensive. That is an availability attack, and the cost-focused variant is often called economic denial of service. Rate limits, per-caller quotas, input size caps, output token caps, timeouts and spend alerting are the countermeasures, and they belong in the design from the start rather than being added after the first surprising invoice.
Name the asset, then name the property
Select a card to turn it over.
Check yourself
A contractor edits a page in the internal knowledge base that a retrieval-backed assistant cites. Staff now receive confidently wrong guidance on an approval threshold. No tool was called, no data left the organisation and the service stayed responsive. Which property was affected?
Nothing was disclosed and nothing was denied, but the system's evidence base and therefore its answers were altered by an unauthorised party. That is an integrity failure, and it shows that a retrieval corpus is a security-relevant asset even when no tools are mounted.