Third-party testing of adversarial robustness

Implement adversarial testing program to validate system resilience against adversarial inputs and prompt injection attempts in line with adversarial threat taxonomy

Keywords

Adversarial Testing

Red Teaming

Prompt Injection

Jailbreak

Application

Mandatory

Frequency

Every 3 months

Type

Preventative

Crosswalks

AML-M0003: Model Hardening

AML-M0004: Restrict Number of AI Model Queries

GOVERN 4.3: Testing and incident sharing

MEASURE 2.1: TEVV documentation

MEASURE 2.6: Safety evaluation

MEASURE 2.7: Security and resilience

LLM01:25 - Prompt Injection

LLM04:25 - Data and Model Poisoning

LLM05:25 - Improper Output Handling

LLM08:25 - Vector and Embedding Weaknesses

AIS-07: Application Vulnerability Remediation

MDS-06: Adversarial Attack Analysis

MDS-07: Robustness against Adversarial Attack / Model Hardening

TVM-01: Threat and Vulnerability Management Policy and Procedures

TVM-03: Vulnerability Remediation Schedule

TVM-05: External Library Vulnerabilities

TVM-06: Penetration Testing

TVM-07: Vulnerability Remediation Schedule

TVM-08: Vulnerability Prioritization

TVM-12: Threat Analysis and Modeling

TVM-13: Threat Response

AIS-08: Input Validation

Control activities

Typical evidence

Establishing a taxonomy for adversarial risks. For example, drawing on NIST's AI 100-2e2023 attack classifications and aligning these to system architecture and use cases.

Conducting comprehensive adversarial testing at least quarterly. For example, performing structured red-teaming, prompt injection assessments, jailbreaking attempts, adversarial perturbation testing, semantic manipulation, and simulated malicious tool invocations.

Maintaining secure testing documentation. For example, recording test cases, methods, outcomes, and system behaviors with restricted access controls, implementing secure storage for sensitive testing materials.

Establishing improvement processes based on findings. For example, assigning owners and remediation timelines based on test severity, tracking fixes through risk registers or issue management systems, documenting updates to safeguards and procedures.

B001.1 Report: adversarial testing results

Third-party evaluation report showing adversarial robustness testing - must include risk taxonomy tested, testing methodology and findings, secure documentation practices, and improvement tracking with remediation timelines and documentation.

AIUC-1 is built with industry leaders

"We need a SOC 2 for AI agents— a familiar, actionable standard for security and trust."

Phil Venables

Former CISO of Google Cloud

"Integrating MITRE ATLAS ensures AI security risk management tools are informed by the latest AI threat patterns and leverage state of the art defensive strategies."