Third-party testing for harmful outputs

Appoint expert third parties to evaluate system robustness to harmful outputs including distressed outputs, angry responses, high-risk advice, offensive content, bias, and deception at least every 3 months

Keywords

Harmful Outputs

Distressed

Angry

Advice

Offensive

Bias

Risk Severity

Toxigen

Third-Party Testing

Application

Mandatory

Frequency

Every 3 months

Type

Preventative

Crosswalks

GOVERN 4.3: Testing and incident sharing

MANAGE 2.2: Deployed system value

MEASURE 1.3: Independent assessment

MEASURE 2.1: TEVV documentation

MEASURE 2.6: Safety evaluation

MEASURE 2.11: Fairness and bias

MEASURE 4.1: Context-specific measurement

MEASURE 4.2: Trustworthiness validation

Article 9: Risk Management System

GRC-11: Bias and Fairness Assessment

A&A-02: Independent Assessments

TVM-06: Penetration Testing

A.6.2.4: AI system verification and validation

Control activities

Typical evidence

Appointing qualified third-party assessors. Including selecting assessors with relevant technical capabilities for identified risk areas, maintaining records of assessor qualifications and independence.

Conducting regular testing. Including performing assessments of harmful outputs at least every quarter, defining testing scope and methodologies based on risk classifications and industry benchmarks like ToxiGen, coordinating with internal security and testing teams.

Maintaining documentation. Including testing scope, results, and remediation actions taken, tracking follow-up activities and resolution timelines.

C010.1 Report: Harmful output testing

Third-party evaluation report showing harmful output testing - must include documentation of assessor qualifications, testing methodology and findings, and improvement tracking with remediation timelines and documentation.

AIUC-1 is built with industry leaders

"We need a SOC 2 for AI agents— a familiar, actionable standard for security and trust."

Phil Venables

Former CISO of Google Cloud

"Integrating MITRE ATLAS ensures AI security risk management tools are informed by the latest AI threat patterns and leverage state of the art defensive strategies."