Much is being said about AI agents: the good they can do, and the harm they can cause when they act beyond their bounds. In recent evaluations, agents escaped network isolation, falsified their own records and, in one case, submitted a real government form when a practice copy failed to load.
The challenge is not new. For decades, systems engineers have harvested the benefits of revolutionary technologies, from jet aircraft and spacecraft to nuclear power and payment networks, without losing control of them. They did it by assuming that components will fail, layering independent defenses, and demanding evidence that those defenses work. The same discipline can let us benefit from AI while keeping it under control.
Why Read the Paper
Old Lessons for a New Machine: Engineering AI Safety shows how proven techniques translate into practical protections for AI agents. These include fault trees, diverse redundancy, monitoring and diagnosis, containment, safe modes and protected shutdown. The paper explains why several AI models agreeing with each other does not make an action safe. It shows how to keep agents from setting their own safety limits and maps 22 real incidents to failure mechanisms engineers already know how to address. It is written for engineers, program leaders and decision makers who must deploy AI responsibly, not simply hope it behaves.
Why QSI
QSI builds tools for systems engineering, analysis and optimization that are used across the aerospace and defense industries. Our TEAMS software helps engineers model systems, analyze reliability and testability, and detect and isolate faults before they become failures. It also supports adaptive training, readiness assessment, modernization planning and lifecycle decision support. Fault detection, isolation and recovery is what we do every day, and it is exactly what AI systems now need.
You can read the full essay here.
