Satya Nadella outlined the principles for the safe use of AI

In recent decades, traditional software systems have become widely adopted in the economy. Their behavior could be traced down to specific code segments.

This approach does not work for modern superintelligence systems. Although advanced AI models already surpass traditional programs in capabilities, their behavior and outcomes cannot be directly linked to specific training data or model configurations. Meanwhile, autonomous systems gain access to confidential information and can perform critically important tasks on behalf of users and organizations.

As AI develops, approaches to the security and control of such systems must be reevaluated. Responsibility for AI actions cannot be fully shifted to model developers. Even provider guarantees do not exempt organizations from the need to independently control the use of the technology.

Superintelligence should not be viewed as a set of black boxes whose recommendations and actions are merely accepted or rejected. It is necessary to create systems with limited authority, whose behavior can be monitored, constraints can be verified, and actions can be halted if necessary.

In other words, the provision of intellectual capabilities should be separated from the authority to manage them.

How to Control AI

Setting aside the complex problem of aligning AI goals with human values, security should be based on an engineering approach to limiting capabilities and managing systems. Unpredictable models must be surrounded by a robust architecture with clearly defined rules, human oversight mechanisms, and proven operational procedures. Where existing requirements are insufficient, industry standards should be developed.

One possible approach is to treat advanced models with both closed and open weights as potential sources of internal threats. This does not mean that AI necessarily acts with malicious intent. However, any sufficiently powerful system with access to critical resources may make errors or become compromised. Security architecture must account for this possibility.

Organizations already have experience managing internal threats. Over decades, the information security field has developed practices that help control access to corporate systems. These include identity verification, access restriction, action logging, and isolation of individual components.

Now, these principles are beginning to be applied to superintelligence systems within organizations. One of the first steps is ensuring transparency in the model's reasoning chain. The specifics of AI's internal language should not serve as an excuse for the opacity of its logic.

However, reasoning transparency alone is insufficient. There is currently no reliable way to guarantee that a model's results always accurately reflect its reasoning process and allow for a complete understanding of how an answer was derived.

Models can be used to test each other and check for resilience against attacks. But this approach does not solve the problem if one opaque model operates within an opaque management system, while another model with similar limitations observes it. The result is several nested black boxes, whose security is difficult to verify independently.

Therefore, mechanisms determining a model's access to data and permitted actions must reside outside the model itself. This approach is based on an information security principle formulated in the 1970s: a program should not have the ability to bypass or alter the mechanisms that enforce its access rights.

For modern AI systems, this means the necessity of separating the model from the management environment that coordinates its operation and from the set of actions it is allowed to perform. Control and protection mechanisms must also be independent of the model itself.

How to Ensure AI Security

Secure artificial intelligence systems should be built on several key principles:

Model Diversity. The outcome of an important task should not depend exclusively on a single model. Nor should it rely solely on the model to verify its own work.

Action Observability. Every significant action taken by a model must leave tamper-proof evidence available for human analysis. If an action cannot be tracked, it is harder to trust. Organizations must be able to reconstruct the sequence of actions that led to a specific result without relying on explanations from the model itself.

Verifiability. The entire system, including its behavior during failures, attacks, non-standard situations, and configuration changes, must be regularly tested. Checks should not be limited to cases where the task is completed successfully.

Independent Control. Organizations must independently determine which data and resources a model can access and which actions it is permitted to perform.

Independent Audit. Security verification must be conducted independently of the AI whose performance is being evaluated. One model should not simultaneously manage system behavior and control evidence that its actions align with initial goals.

Isolation and Halt Capability. Design must account for the risk of model compromise and pre-emptively provide mechanisms for its isolation. Authorized personnel must be able to pause the model's operation or completely shut it down at any time, including during task execution. As AI advances, more sophisticated isolation technologies and unified industry standards for their application will be required.

Incident Disclosure. In the event of a system failure or compromise, affected parties must receive timely information about the incident. Organizations must exchange information about the causes of incidents, failed protection mechanisms, and methods for preventing similar situations. This exchange should cover the entire industry and include technical details affecting the behavior of autonomous agents during operation.

The reliability of a superintelligence system should not depend exclusively on the level of trust in the model itself. The most secure system is one that minimizes reliance on this trust through independent control, limitation of authority, and the possibility of human intervention.