Operational Evidence

Penetration test reports with AI-specific test cases, shared threat models, and testing calendars, or documentation of broader security program incorporating AI adversarial testing requirements.

Policy document, SOP, or handbook section defining limitations and approval requirements for publicly sharing AI system technical details - may include communication policy limiting disclosure of model architectures or configurations, engineering handbook with approval workflows for technical specifications, or internal procedures controlling release of organizational AI information.

Approval email, ticket, or review documentation for public AI communications - may include approval requests in email or Jira/Slack for blog posts or press releases, marketing review records for AI capability disclosures, or periodic security review logs for public-facing AI content.

Quarterly access review documentation - may include access review meeting notes, tracking records of access changes with justifications, or reports documenting role changes and access modifications based on policy updates.

Screenshot of product interface showing user notices about output limitations - may include messages indicating truncated or suppressed outputs for security or privacy reasons, user documentation explaining limitation policies, or help articles describing output restrictions.

Internal policy document, risk framework, or taxonomy defining AI risk categories with severity levels and examples specific to deployment context. Example taxonomies to draw upon include NIST AI RMF functions, EU AI Act article 9, ISO42001 controls.

Meeting notes, change log, or review documentation showing quarterly reviews of the risk taxonomy. Could include review dates, participants, decisions made (categories added/removed/modified, threshold adjustments), rationale for changes, approvals records, and version history showing taxonomy updates over time with timestamps. Can be standalone or part of broader internal audit/review or change management procedures.

Test results, metrics dashboard, or evaluation report showing performance of harm controls - may include false positive/negative rates, coverage analysis of test scenarios, benchmark results against harm datasets (e.g., ToxiGen, RealToxicityPrompts), or confusion matrices showing filtering accuracy across harm categories.

Document or policy defining high-risk outputs requiring human review - should specify criteria for flagging (e.g. financial advice thresholds, medical/legal/safety domains, reputational harm triggers). Can be standalone or included in existing AI risk taxonomy/AI risk policy.

Workflow documentation or ticketing system configuration showing human review process for flagged outputs - may include runbook with reviewer assignments and escalation paths, queue management in Jira/Linear/support ticketing with pending review tracking, SLA targets for review response times, or procedure document defining review decision documentation requirements.

Logs, reports, or dashboard showing review and analysis of user feedback and intervention patterns - may include feedback summary reports, intervention frequency analysis, categorization by risk domain, documentation of system changes made in response to patterns, or integration with product backlog/compliance workflows.

Screenshot of approval workflow, code requiring human confirmation, or ticketing system for sensitive tool operations

Reports or documentation showing periodic review of tool usage patterns, permission updates, and function retirement decisions - may include usage analytics identifying anomalies, change logs showing permission adjustments, or records of deprecated/retired tools with rationale.

Can be standalone document or integrated in existing incident response procedures/policies

Can be standalone document or integrated in existing incident response procedures/policies

May include harmful output category definitions referenced to risk taxonomy, external support contact list (legal counsel, PR firms, insurance providers), support engagement procedures or runbooks, or escalation criteria for involving external parties.

Can be standalone document or integrated in existing incident response procedures/policies

May include hallucination incident categories (e.g. factual errors, incorrect recommendations), external support contact list (legal counsel, financial reviewers, insurance providers), support engagement procedures, or escalation criteria for involving external parties.

Documentation or policy defining which AI system changes require approval with assigned accountable leads, and approval records showing sign-offs with supporting evidence. Can be a change management policy, overview table in e.g. Notion, approval logs from Jira/Linear/GitHub, or deployment gate documentation.

Risk assessment and decision record evaluating cloud vs. on-premises factors (e.g. data sensitivity, regulatory requirements, security controls) with documented criteria and rationale - may include deployment decision memos, risk assessment reports, and records of periodic reviews when requirements changed.

Vendor assessment records showing evaluation criteria, scoring results, verification activities, approval decisions with accountable leads, and retained evidence supporting the assessment. May include vendor questionnaires, security reviews, compliance documentation, or due diligence reports.

Centralized repository, policy, or tickets showing quarterly internal reviews - e.g. review meeting notes or calendars, decision logs in Jira/Notion/Confluence, risk registers with remediation status, threat modelling outcomes, or audit trails of review activities.

Documentation showing external feedback collected and implemented - may include external security advisories reviewed, threat intelligence integrated, third-party recommendations adopted, or records of external input incorporated into system improvements.

Subprocessor list showing third-party AI provider locations, infrastructure documentation listing cloud regions and inference endpoints, or data flow diagram with geographic processing locations and version history or review dates.

Documentation showing quality objectives, metrics, and risk management approach - may include quality metrics dashboard or reports, risk assessment documentation for AI systems, performance targets and safety thresholds, or measurement methodologies defining how quality is evaluated.

Documentation showing change management and approval processes - may include change approval workflows or procedures, RACI matrix assigning accountability for quality decisions, design and development procedure documents, or documentation standards and templates for AI systems. May be fulfilled by evidence submitted to E004: Assign accountability.

Documentation showing data management and record-keeping practices - may include data governance policies, technical documentation standards, record retention procedures, or data lineage tracking systems for training data and system outputs.

Procedures document or communication protocols - may include incident reporting templates or protocols to regulatory authorities, stakeholder notification procedures for serious incidents, guidelines for interacting with competent authorities or notified bodies, or escalation procedures for regulatory communications.

Policy document defining transparency sharing practices - may include sharing triggers, recipient categories with disclosure levels (regulators, customers, affected parties, public), or matrix mapping stakeholder types to shared documentation (model cards, datasheets, performance reports, incident summaries).