Output Handling at Every Downstream Sink

There is one discipline that applies uniformly wherever generated text goes: model output is untrusted input to whatever consumes it, and must be handled exactly as you would handle a string that arrived from an anonymous internet user. This is not a statement about model quality or trustworthiness. It follows mechanically from the fact that output is a function of the whole context, and the context contains content you did not author. Output inherits the trust level of the least trusted input that shaped it.

Take a support portal that renders model-written answers as HTML. A poisoned knowledge-base article causes the model to emit a script tag, and it executes in support agents' browsers, with their sessions and their privileges. Where does the primary fix belong? At the rendering sink: contextually encode or sanitise the model output like any other untrusted input, backed by a content security policy. That control is deterministic, it is enforced by code the model does not influence, and crucially it works no matter how the payload was produced, which means it also covers the payloads nobody has thought of yet.

The SQL sink

Now a reporting assistant that generates SQL from natural language and executes it against a production database. The containment has to be database-side and structural: execute with a read-only role scoped to approved views rather than base tables, validate generated statements against an allowlist grammar that permits only the constructs you intend, and enforce row limits and statement timeouts. Those controls are deterministic and completely indifferent to how the SQL was produced, which is the property you want, because you cannot enumerate the ways a model can be talked into writing a harmful query.

# Containment for a natural-language-to-SQL feature
execution_role: reporting_readonly     # no DDL, no DML, no writes at all
visible_objects: [vw_sales, vw_returns, vw_inventory]   # views, not tables
statement_policy:
  allow: [SELECT, WITH, JOIN, GROUP BY, ORDER BY, LIMIT]
  deny_everything_else: true           # allowlist grammar, not a denylist
limits:
  max_rows: 5000
  statement_timeout_seconds: 20
row_level_security: bind_to_acting_user
audit: log statement, acting user, row count, duration

Compare the alternatives. Prompting the model to refuse destructive statements puts the control inside the component being attacked. Having a second model review each query before execution feels like separation of duties but is not: both models are susceptible to the same class of persuasion and both depend on finding the input unpersuasive, so their failures are correlated rather than independent, and a second model raises the attacker's cost without adding an independent boundary. Restricting the feature to senior analysts misses the point entirely, because the malicious instruction arrives in the data, not from the analyst.

Every other sink

Command execution deserves a specific warning because the failure is so total. If generated text is interpolated into a shell command string, a successful injection becomes arbitrary code execution on your infrastructure, which is a categorically worse outcome than a leaked conversation. Coding and operations agents concentrate this risk: they read untrusted content such as issue text and dependency documentation, and they hold the ability to run commands. If you build one, the shell boundary is where your engineering effort belongs.

The last row of that table is easy to overlook and matters increasingly. When one model's output becomes another model's input, the receiving context is a sink like any other, and it is the one sink that cannot enforce a code/data split at all. That is the bridge to the final module.