AI safety is an
engineering problem.

01 / System boundaries

Safety in the system.

Safety is not a property of the model alone. It is a property of the system the model runs inside.

Aviation, nuclear power, and distributed systems build safety under uncertainty. They manage it continuously through failure modes, margins, and monitoring. That is a useful engineering frame for AI systems too.

Rate limits, human approval gates, blast-radius containment, and least privilege have established precedents. Output classifiers and prompt-injection defenses address a newer, probabilistic component under adversarial pressure. Those controls need their own scrutiny; they cannot be assumed to offer the same guarantees.

02 / Agent alignment

Make the boundaries concrete.

For an agent, alignment becomes an architectural question: which goals, permissions, and actions can the surrounding system actually enforce?

A model can propose an action without being authorized to execute it. Deterministic workflows, explicit tool permissions, sandboxing, and approval gates give that distinction a concrete implementation. Memory and behavioral rules also need to be inspectable, so an agent’s account of what happened can be checked against evidence.

This does not settle alignment at the model level. It gives engineers specific boundaries to design, observe, and revise when behavior falls outside the intended task.

03 / Security & controls

Design for control failures.

Some failures are familiar: write access granted without blast-radius thinking, or irreversible operations performed without a dry run. Others arise from probabilistic outputs and adversarial inputs, including prompt injection. Both belong in the system’s failure analysis.

• Constrain access and action to the task’s required scope.
• Contain execution and require approval where the consequences warrant it.
• Observe the actions taken and retain evidence of their results.
• Revisit semantic defenses as behavior and adversarial pressure change.

04 / Incentives & evidence

Reward useful, verifiable work.

Per-token pricing pays for volume rather than utility. Training can reward pleasing answers over precise ones. Those incentives matter alongside the technical controls: a system needs an honest measure of successful work, not just activity.

Engineering discipline and incentive design belong together. The practical question is whether the system performs the intended task, within its boundaries, with results we can verify.