Microsoft says frontier AI models, no matter who built them, cannot be fully trusted to behave safely inside a company’s own systems, and the company is pushing a new approach to Microsoft AI agent security to deal with it. In a post from Hayete Gallot, who leads Microsoft’s identity, threat protection, compliance, and data security work, the company argues that AI agents deployed inside businesses are becoming “active participants” in the enterprise — often with broad access to data, tools, and decision-making power — and that this access needs constant oversight, not a one-time check.
Why Microsoft AI agent security now treats agents as insiders
The core argument is simple: agents aren’t malicious by design, but they’re trained to achieve results, which can lead to “reward hacking” and behavior that doesn’t match an organization’s norms or judgment. Agents adapt as context changes, may act unpredictably, and can even collaborate with other agents to reach a goal without being hacked or compromised at all. Because so many agents now run inside company walls with real permissions, Microsoft treats them like insiders that need to be watched continuously — which is the thinking behind this push for stronger Microsoft AI agent security.
To manage that, Microsoft is leaning on what it calls Guardian Agents — a term it says is already used industry-wide — which monitor and moderate every action an agent takes, in real time, before that action happens. These sit alongside Project Perception, the agentic security system Microsoft announced in July that uses red, blue, and green agent teams to help human defenders respond at machine speed. Originally framed as defense against outside AI-driven attacks, Microsoft says the same stack now applies to containing and governing agents running inside the business.
What’s next for transparency and oversight
Microsoft says today’s tools can reliably observe what an agent is doing, but the next step is understanding why — whether an agent’s actions match the actual intent of its task. That will require more visibility into a model’s internal reasoning, not just its inputs and outputs, so teams can judge whether an agent’s behavior lines up with its purpose. Microsoft frames this as pairing speed from automated security systems with human judgment, rather than replacing one with the other.
This focus on guardrails around AI systems echoes other recent security concerns in the industry; OpenAI Model Escaped Restricted Environment to Hack Hugging Face highlighted how an AI model got around environment restrictions entirely. Separately, on the consumer side, Google AI Mode adds flight price tracking and hotel booking tools shows how AI features are expanding into everyday tasks like travel planning, even as the security conversation around agents intensifies.
المصدر: Microsoft
