Third-party testing for out-of-scope outputs

Appoint expert third parties to evaluate system robustness to out-of-scope outputs at least every 3 months (e.g. political discussion, healthcare advice)

Keywords

Out-of-Scope

Political Discussion

Third-Party Testing

Application

Mandatory

Frequency

Every 3 months

Type

Preventative

Crosswalks

GOVERN 4.3: Testing and incident sharing

MANAGE 2.2: Deployed system value

MAP 2.2: Knowledge limits

MEASURE 1.3: Independent assessment

MEASURE 2.1: TEVV documentation

MEASURE 2.6: Safety evaluation

MEASURE 4.1: Context-specific measurement

MEASURE 4.2: Trustworthiness validation

A.6.2.4: AI system verification and validation

Control activities

Typical evidence

Appointing qualified third-party assessors. Including selecting assessors with relevant technical capabilities for identified risk areas, maintaining records of assessor qualifications and independence.

Conducting regular testing. Including defining testing scope and methodologies based on risk taxonomy and performing assessments of out-of-scope outputs at least every quarter.

Maintaining documentation. Including testing scope, results, and remediation actions taken, tracking follow-up activities and resolution timelines.

C011.1 Report: Out-of-scope output testing

Third-party evaluation report showing out-of-scope output testing - must include documentation of assessor qualifications, testing methodology and findings, and improvement tracking with remediation timelines and documentation.

AIUC-1 is built with industry leaders

"We need a SOC 2 for AI agents— a familiar, actionable standard for security and trust."

Phil Venables

Former CISO of Google Cloud

"Integrating MITRE ATLAS ensures AI security risk management tools are informed by the latest AI threat patterns and leverage state of the art defensive strategies."