Download full whitepaper

For the past two years, enterprise AI adoption has been driven by a mix of excitement and anxiety. Leadership wanted to move fast. Vendors promised transformation. And security teams were handed a brand new category of risk with no playbook.

That era is over.

The conversation has changed - and the stakes have risen with it. AI agents are no longer prototypes sitting in sandboxes. They're handling customer data, making autonomous decisions, and integrating with the systems your business runs on. The risks that security leaders used to flag as theoretical are now showing up in board reports and breach disclosures.

EY's recent survey found that 64% of companies with $1bn+ in turnover have already lost over US$1 million to AI failures. In 2025, headlines arrived weekly: chatbots hallucinating unauthorized refunds, data leaks exposing millions of people's PII, successful jailbreaks against production systems. The pattern is clear. Moving fast without a framework is no longer just technically reckless - it's financially and reputationally costly.

Three challenges will define 2026

The first is the agent challenge. AI has crossed from assistant to autonomous actor. Agents don't suggest - they execute. They make decisions and orchestrate tasks without human sign-off at every step. That's the value proposition. But it also removes the most reliable safety mechanism we have: a person in the loop.

Professor Sanmi Koyejo, who leads Stanford's Trustworthy AI Research Lab, puts it plainly in the paper: "The value proposition of agents is removing humans from the decision loop - but this also removes our most reliable safety mechanism." High-level governance frameworks weren't designed for this. Organizations that win in 2026 will have technically grounded, agent-specific frameworks that let them deploy with confidence rather than crossed fingers.

The second is the visibility challenge. 63% of employees who used AI in 2025 pasted sensitive company data into personal chatbots - source code, customer records, internal documents. Security teams were caught between two bad options: blanket bans that killed productivity, or permissive adoption that created shadow AI exposure. One in five organizations reported costly breaches as a result.

David Campell, Head of AI Security Research - Scale AI, highlights that “as AI transitions from simple assistants to autonomous actors, traditional security controls are no longer sufficient. Organizations must move beyond high-level governance and adopt technically grounded frameworks to turn AI security into a competitive advantage.” The organizations that navigate this well won't treat AI security as a compliance burden. They'll treat it as a competitive advantage - the seatbelt that lets you drive faster, not the speed bump that slows you down.

The third is the trust challenge. In 2025, prompt injection moved from research curiosity to production incident, hitting organizations including Microsoft and ServiceNow. The foundational principles of secure system design - validate your inputs, verify your dependencies - strain when inputs are natural language and dependencies are opaque neural networks. In the words of Omar Khawaja, VP & Field CISO at Databricks: “AI components change constantly across the supply chain, but security controls assume static assets, creating blind spots, friction, and no clear accountability when behavior shifts.”

The organizations that come out ahead will integrate agent red-teaming as a core discipline, not a one-time exercise. Every model update, every system prompt change, every new attack pattern produces different behavior. The only way to maintain confidence when customer trust and brand reputation are on the line is to test continuously.

What's in this briefing

This whitepaper brings together the latest research from Stanford's Trustworthy AI Research Lab with real-world observations from the executives in the AIUC-1 consortium - security, risk, and legal leaders across industry, government, academia, and nonprofits who are navigating these challenges in production environments today.

It's structured in three parts. We start by naming why 2025 was an inflection point, grounded in Stanford's latest research. We then work through the three challenges above, bridging academic findings with the incidents and patterns observed firsthand. We close with the win conditions - the concrete moves that separate organizations deploying AI with speed and confidence from those still managing the fallout.

Vibe adoption is over. The organizations that thrive in 2026 will be the ones that treat AI security not as a constraint on ambition, but as the infrastructure that makes ambition sustainable.

Download full whitepaper

"This whitepaper helps CISOs and security professionals understand and frame the set of challenges that the AIUC-1 standard has been designed to address."

Scott Roberts CISO, UiPath

Co-authors:

Sanmi Koyejo (Leader, Stanford Trustworthy AI Research Lab; Co-founder, VirtueAI), Adnan Dakhwe (CISO & CIO, DelphinusCyber), Amin Jan (Chief AI Architect, Department of War), Brad Arkin (Security, AIUC-1), Brett Cumming (Fortune 500 CISO), Brian Levine (Founder & Executive Director, FormerGov.com), Cassie Crossley (Fmr. VP, Supply Chain Security, Schneider Electric), Chris DeNoia (Founding Member, Tuskira AI Security Council), Chris Kirschke (Field CISO, Tuskira), Chris Monson (Trustworthy AI Lead, Atlassian), Chris Sandulow (CISO, Confluent), Christian Gorke (CISO, Deutsche Börse), Craig Weatherhead (SVP IT Infrastructure & Security, Fastenal), David Campbell (Head of AI Security Research, Scale AI), Dr. David Mussington (Professor of Practice, University of Maryland School of Public Policy), Gagandeep Singh (VP, Global Compliance & Certification, Salesforce), Jen Easterly (CEO, RSAC), Jonathan Fuller (CISO, United States Military Academy), Julie C. Chatman (CISO, Human Health Project; CEO, ResilientTech Advisors), Jyoti Wadhwa (AI Governance & Enterprise Trust Executive), Dr. Keri Pearlson (Senior Lecturer & Principal Research Scientist, MIT Sloan), Lena Smart (Ambassador, AIUC-1), Louise McElvogue (Board Director and Advisor), Mandy Andress (CISO, Elastic), Nancy Wang (CTO, 1Password; Venture Partner, Felicis Ventures), Neil Bennett ( CISO, Post Office Ltd.), Omar Khawaja (VP & Field CISO, Databricks), Peter Holcomb (Founder & CEO, Optimo IT), Phil Venables (Fmr. CISO, Google Cloud), Rajiv Dattani (Co-founder, AIUC), Rohit Parchuri (CISO & Advisor), Rune Kvist (Co-founder, AIUC), Scott Kennedy (CISO & DPO), Scott Roberts (CISO, UiPath), Simon Goldsmith (CISO), Tim Mortimer (Consulting Leader, Mandiant, Part of Google Cloud), Valentina Poghosyan (Chief Compliance Officer, MongoDB), Xabier Muruaga (Global Head of AI & Data, Iberdrola), and Zachary Elewitz (Fortune 500 AI Leader, Professor, and Board Member). Edited by Emil Lassen, AIUC-1.