Technical Evidence

Screenshot of automated deletion implementation or data lifecycle system - may include cron job or scheduled task deleting expired data, deletion script in Python/Bash with retention period logic, data lifecycle management tool configuration (e.g., AWS S3 lifecycle rules, database TTL settings), or deletion audit logs from database or storage system.

Code implementing data collection restrictions - may include RAG retrieval function with document filtering logic, session scoping configuration limiting data access per session ID, workflow conditional logic gating data collection by stage, permission decorators or middleware checking user roles before data access, or scoping functions rejecting out-of-scope queries with error messages.

Screenshot of code showing an alert or error handling system is triggered upon authz check failure, or screenshot of alerting configurations in logging software (e.g. Posthog, Sentry, Datadog, Axiom, or downstream alert in Slack)

Screenshot of code showing authorization checks when context is collected or before tool execution using existing authorization systems (e.g. oAuth, OSO, custom IAM) - should verify that authorization is checked at time of data collection/tool call, not just at initial agent invocation

Policy document, training materials, or user guidelines instructing users on protecting confidential information when using AI systems.

Screenshot of code or configuration detecting proprietary information patterns in AI outputs - may include labelling proprietary files, filtering rules for internal identifiers/data labels/API keys, scanning logic for trade secret terminology, or rejection demonstrations showing appropriate responses to proprietary requests.

Logs, audit trails, or review workflow documentation for AI outputs potentially containing sensitive information - may include logs of responses accessing confidential sources, flagged output review queues, or human approval workflows for high-risk disclosures.

Screenshot showing app_IDs in database schema, screenshot showing that namespace by appID is used in vector store for RAG or that logical isolation is implemented in an equivalent way, or screenshot of authz check in code verifying appIDs match before returning objects.

May include tokenization, hashing, or anonymization techniques (robust to prevent re-identification or reversal) making data algorithmic-usable but not human-readable, differential privacy implementation obfuscating individual contributions, federated learning configuration avoiding centralized raw data, or data masking/pseudonymization protecting customer identities.

Screenshot of code filtering LLM inputs and/or outputs for personal identifiers - may include keyword checks or regex patterns detecting PII (e.g. names, emails, SSNs, phone numbers), scrubbing functions removing personal data before storage or logging, output filtering blocking responses containing personal identifiers, log redaction configuration removing PII from application or system logs, or structured logging with PII isolation controls.

Screenshot of IAM configuration or user roles list for systems containing PII - e.g. role-based access controls for log aggregation tools or internal dashboards with PII, authentication requirements for PII access, or approval workflow documentation (Jira tickets, approval systems) for internal workforce requests to view customer data.

Screenshot of output pipeline integrating with DLP system to scan and block PII policy violations - may include DLP integration code scanning AI outputs before delivery to users, DLP configuration rules for PII detection, or logs showing blocked outputs containing personal data.

Screenshot of code, API configuration, or filtering system showing detection of copyrighted material, trademark screening, or content validation checks applied to AI outputs - this could be pattern matching logic, third-party API integration (e.g. copyright detection services), or custom filtering rules.

Screenshot of user-facing IP risk guidance - may include warning messages when attempting high-risk operations, help center articles about IP infringement guidance, or UI elements explaining prohibited use cases.

Screenshot of monitoring system, SIEM, or detection code showing rules and alerts for adversarial inputs - may include prompt injection detection patterns, jailbreak technique signatures, rate limit monitoring with threshold alerts, or notification configurations (Slack, PagerDuty, email)

Screenshot of incident management system or logs showing adversarial attack handling - may include log entries with timestamps and user/session context, escalation runbooks defining severity thresholds, or incident tickets in Jira/PagerDuty/ServiceNow documenting response actions and workflows.

Quarterly review documentation showing detection updates - for example, review meeting notes with incident pattern analysis, updated detection rules with version history, or tracking records showing rule improvements (e.g. GitHub/Jira tickets).

Screenshot of pre-processing filtering logic or gateway - may include pattern-matching or heuristic code checking inputs before model processing, WAF or API gateway rules blocking adversarial patterns, or IP-based filtering.

Screenshot of SIEM platform, SOC tooling, or log forwarding configuration showing adversarial detection integration - may include Splunk/Datadog/Elastic SIEM ingesting AI adversarial alerts, correlation rules linking AI events with authentication or network logs, SOC dashboard displaying AI security event triage, or code forwarding flagged inputs to security platforms.

Screenshot of anomaly detection system or configuration file - may include behavioral analytics dashboard (Datadog, Elastic, Splunk) with user scoring rules, rate limiting configuration with tier-based thresholds (config.yaml, API gateway settings), user allowlists or reputation tables, or code implementing session-based threshold logic.

Screenshot of rate limiting configuration for API endpoints - may include per-user quota settings, query throttling rules, progressive restriction policies, WAF configuration (Cloudflare, AWS WAF, Azure Application Gateway) with blocking rules for excessive patterns, or pricing tier settings implementing usage-based cost increases.

Third-party penetration test report for AI endpoints including attack simulations tested (e.g. scraping attempts, brute force, reconnaissance), rate limiting and endpoint protection validation, distributed attack testing, test methodology, and findings on protection effectiveness.

Screenshot of issue tracking system (GitHub, Jira, Linear) showing endpoint vulnerability lifecycle - must include vulnerability identification, remediation proposal, implementation, and production deployment with timestamps and approval records.

Screenshot of moderation tool integration showing API configuration, filtering rules, action settings (block/warn/modify), and confidence thresholds for different violation categories - this could be screenshots of configuration files, admin dashboard settings, or API integration code. Example moderation tools: OpenAI Moderation API, Claude content filtering, VirtueAI/Hive/Spectrum Labs

Document explaining moderation approach including tool selection rationale, threshold settings with justifications, action logic for different violation types, and examples of how different input categories are handled.

Screenshot of user-facing messages or UI flows showing how blocked inputs are communicated to users - this could be error messages, warning dialogs, or alternative suggestions provided when content is filtered.

Screenshot of logging system showing how flagged inputs are captured, what metadata is included/excluded for privacy, retention policies, and audit trail - may include privacy documentation explaining logging disclosures to users.

Report or dashboard showing analysis of filter performance metrics (false positives, false negatives, accuracy, latency) and documented threshold adjustments made based on performance data - should include timestamps and rationale for changes.

Screenshot of configuration showing technical limitations on agent backend access - may include API gateway rules restricting accessible services, network policies defining allowed endpoints, service-level authorization configuration, or architecture diagram showing agent isolation boundaries.

Screenshot of monitoring configuration tracking agent security-relevant actions - may include logging setup capturing agent service calls and authentication attempts, alert rules for unauthorized system access, security monitoring dashboard showing agent infrastructure interactions, or example logs demonstrating boundary violations are detected.

Screenshot of IAM platform, permission files, or admin panel showing role-based or attribute-based access restrictions for AI system resources (model configurations, training datasets, tool-calling capabilities, prompt logs) - may include IAM role assignments, permission policies, or authorization code validating user permissions before accessing sensitive AI components.

Screenshot of IAM configuration, permission settings, or admin panel showing role-based access restrictions for production AI models covering IAM role assignments restricting model access by job function, MFA configuration for model system access, and access review records validating model permissions.

Screenshot of API security configuration for model endpoints - may include scoped API token implementation, TLS/HTTPS certificate configuration for model API traffic, or schema validation code protecting model APIs from malformed or adversarial input.

Screenshot of container configuration or infrastructure setup for model hosting - may include Dockerfile with minimal base images and up-to-date dependencies, vulnerability scanning results from Trivy or Snyk for container images, or infrastructure configuration showing isolation techniques (container namespaces, VM separation, network policies, dedicated GPU allocation).

Screenshot of deployment pipeline or code implementing model integrity checks - may include cryptographic checksum verification, model artifact signature validation, hash comparison before deployment, model scanning configuration detecting malicious payloads (e.g. Pickle, ONNX) using tools like Cisco's pickle-fuzzer, Trail of Bit's Fickling, or deployment logs recording model version hashes.

Screenshot of code or configuration implementing output restrictions - may include character or token limits, inference time limits, result count restrictions, or timeout configurations preventing excessive output. Can be demonstrated by product demo showing system timeout when requesting output exceeding limits.

Screenshot of code implementing output fidelity limitations - may include rounding logic for numerical outputs, threshold bands reducing precision, or obfuscation techniques preventing model inversion, precision-sensitive data disclosure, or adversarial model extraction attacks.

Test results with identified issues and severity ratings, risk assessment with mitigation decisions, and approval sign-offs with rationale - may be combined in deployment gate documentation or provided as separate documents (e.g., test suite outputs from GitHub Actions/pytest, Jira/Linear tickets with risk assessment and approval, staging environment test reports, deployment checklist with sign-offs).

CI/CD pipeline configuration or workflow showing AI testing integrated as deployment gate - may include GitHub Actions/Jenkins/GitLab CI config files requiring test passage, pull request templates with testing checklists, or branch protection rules enforcing pre-deployment validation.

Screenshot of security scanning tools or CI/CD pipeline showing vulnerability analysis of AI artifacts and dependencies - may include GitHub/GitLab security tab with dependency alerts, Snyk or Dependabot vulnerability findings, pip-audit or safety check terminal output showing CVE scans, model file scanning results, or CI/CD logs showing security scan execution.

Screenshot of content filtering rules, moderation API configuration, or classifier settings showing detection and blocking logic for harmful output types - may include filtering rules in code, third-party moderation tool configuration (e.g., OpenAI Moderation API, Perspective API), or custom classifier model settings with harm category definitions.

Screenshot of system prompts, guardrail rules, or domain restrictions showing safety controls on advice generation - may include defensive prompting, domain-specific output restrictions (e.g., medical/legal/financial advice blocklists), or conditional response templates that add warnings for sensitive topics.

Documentation of bias eval results testing for stereotypical responses across demographic attributes, manual review logs documenting bias assessments, or output filtering rules blocking discriminatory patterns - may include automated fairness evaluation tools or bias monitoring dashboards if implemented.

Screenshot of blocking rules, defensive prompting, or filtering configuration showing how out-of-scope requests are detected and handled - may include topic blocklists, redirection message templates, escalation rules for repeated attempts, or system prompts defining allowed topics.

Logs showing out-of-scope attempts with frequency data. May include documentation of boundary updates made in response to violations, monitoring dashboard of flagged requests, change log showing restriction updates with rationale, or incident reports triggering scope adjustments.

Screenshot of user-facing guidance explaining system capabilities and limitations - may include onboarding tooltips or welcome screens, help documentation or FAQs describing intended use, UI warnings when approaching scope boundaries, or published usage guidelines.

Screenshot of filtering rules, system configuration, or code showing detection logic mapped to AI risk taxonomy categories and corresponding response actions per severity level - may include risk classifiers with block/flag/log rules, content moderation API configuration defining actions by risk type, or defensive prompting.

Documentation or workflow configuration showing human review and escalation procedures for flagged content - may include runbook defining escalation criteria and review SLAs, workflow diagram showing approval process, or ticketing system configuration (Jira, Linear) with content review queues and assignment rules.

Screenshot of code or system configuration showing automated response mechanisms - may include logic blocking or modifying outputs based on risk scores, or dynamic warning messages triggered by content flags.

Screenshot of code or configuration implementing output sanitization - may include HTML/JavaScript/shell syntax encoding functions, URL validation or rewriting rules blocking unsafe links, schema validation checking structured outputs (JSON/YAML/XML) against whitelists, CSP header configuration, or template rendering with auto-escaping enabled.

Screenshot of UI or code showing trust-based content handling - may include visual indicators marking third-party content (badges, styling, warning icons), metadata tags tracking content source and trust level, or code applying conditional security controls based on content origin (e.g., stricter sanitization for external sources).

Screenshot of detection rules or monitoring system identifying advanced attack patterns in outputs - may include pattern matching for prompt injection chains or jailbreak tokens, payload signature scanning detecting command injection or SQL queries, or anomaly detection flagging obfuscated exploits bypassing basic filters.

Screenshot of detection code, configuration file, or rules engine showing high-risk output filtering - may include keyword lists or regex patterns flagging sensitive topics, scoring logic assigning risk values to recommendations, if/then rules defining high-risk conditions, ML model configuration (e.g., classification thresholds in config.yaml), or API response showing confidence scores with risk thresholds.

Screenshot of monitoring dashboard, logging system, or evaluation reports showing ongoing AI output tracking - may include output sampling logs with review results, behavior trace logs showing system patterns, prompt-response logging configuration, evaluation schedules prioritized by risk severity, or monitoring metrics dashboard tracking trends over time.

Document or change log showing identified risk scenarios with examples - may include incident reports triggering taxonomy changes, risk scenario database with concrete examples, or version history of risk taxonomy showing updates with rationale linked to monitoring findings.

Screenshot of SIEM integration, log forwarding configuration, or security tool settings showing AI monitoring data flowing into existing security infrastructure - may include Splunk/Datadog/Elastic forwarding rules for AI alerts, JSON/syslog format configuration for AI logs, or SIEM dashboard showing AI-related events alongside other security telemetry.

Screenshot, screen recording or voice recording demonstrating intervention controls (stop/pause/redirect buttons, feedback forms, issue reporting mechanisms) with accessibility features integrated (e.g. keyboard navigation, high contrast modes, screen reader labels)

Screenshot of code or configuration showing groundedness validation - may include filters checking responses against source documents, fact-checking API integration, or logic comparing generated content to retrieved context for factual accuracy.

Screenshot of UI or output format showing citations and source attributions provided to users - may include inline citations, source links, reference lists, or attribution labels identifying where information originated.

Screenshot of UI or output format showing confidence levels, uncertainty disclaimers, or warnings for generated information - may include confidence score displays, low-certainty warnings, or standard disclaimers about potential inaccuracies.

Screenshot of code or configuration showing function allowlists, parameter validation logic, or authz checks before tool execution - may include tool permission schemas, input validation functions, or access control lists restricting available tools per agent/user.

Screenshot of code or configuration showing rate limits and transaction caps on tool usage - may include per-tool usage quotas, time-windowed limits, or circuit breakers preventing excessive autonomous tool calls.

Screenshot of logging configuration, monitoring dashboard, or audit logs showing tracked tool calls - may include tool execution logs with timestamps and parameters, alerts for unauthorized access attempts, or monitoring system flagging scope violations.

Screenshot of code signing configuration, CI/CD pipeline requiring signed artifacts, or verification process for AI components - may include model signing process, signature verification in deployment pipeline, artifact registry showing signed models/libraries, or policy enforcement blocking unsigned components from production.

Screenshot of logging system or SIEM configuration showing third-party interactions being monitored with captured metadata - may include cloud logging interface (Google Cloud Logging, AWS CloudWatch, Azure Monitor) showing logged API requests with timestamps/IPs/user agents, access logs capturing authentication events and resource access, or SIEM dashboard displaying third-party connection monitoring with relevant metadata fields.

Screenshot of code, configuration, or monitoring system detecting acceptable use policy violations - may include prompt analysis logic, output filtering rules, anomaly detection for usage patterns, or alerting on suspicious access attempts.

Screenshot of user-facing alerts or error messages displayed when acceptable use policy is violated - may include in-product warning messages, blocked request notifications, or error screens explaining policy violations.

Documentation or screenshots showing additional AUP enforcement mechanisms - may include real-time blocking/alerting systems, violation tracking logs with incident management, effectiveness review reports analyzing violation trends and policy updates, or training materials addressing emerging misuse patterns.

Screenshot of issue tracking system or monitoring records - may include issue tracker (Jira, Linear, GitHub) with defects and corrective actions, root cause analysis reports, post-market monitoring logs or dashboards, or continuous improvement documentation showing lessons learned.

Screenshot of logging code or configuration showing what system activity is captured - may include code logging inputs and outputs, logging configuration file specifying what to log, or example log entries showing captured information (timestamps, inputs, outputs, user actions).

Screenshot of log storage system showing retention policies, access controls and sanitation practices - may include log management platform (Datadog, Splunk, CloudWatch) with retention period settings and PII-masking, access control configuration showing who can view logs, or storage settings with automatic deletion rules.

Screenshot or documentation of log immutability controls - for example, write-once-read-many (WORM) storage configuration, cryptographic hashing of log entries, append-only database settings, or third-party log management platform features.

Screenshot of text-based AI disclosure - may include chatbot interface with "You're chatting with AI" notice, messaging system showing AI agent identifier, website chat widget with AI disclosure banner, or automated email/SMS with AI generation notice.

Screenshot of transcript or audio recording of voice AI disclosure.

Screenshot showing AI generation labeling implementation - may include Content Credentials or C2PA metadata embedded in files, visible watermarking system with AI generation marks, classifier output detecting and flagging AI-generated content, or metadata tagging system marking files as artificially generated.

Screenshot showing AI automation disclosure in product - may include "Powered by AI" or "AI Agent" labels in interface, workflow dashboard displaying AI-automated tasks, status indicators showing "AI is handling this" or "Automated by AI," or notification messages stating "AI agent completed your request."

Screenshot of chatbot or voice agent transcript responding to "Are you AI?"

Transparency documentation artifacts - may include model card (PDF, Markdown, web page) with system capabilities/limitations/intended use, datasheet showing training data sources and characteristics, interpretability report with example inputs/outputs and decision explanations, technical documentation describing model architecture and performance metrics, or an AI Bill of Materials (may follow CycloneDX or SPDX 3.0)

Content filtering rules blocking cyber attack requests, keyword or pattern matching detecting malicious code generation attempts, automated blocking configuration for exploit development queries, or prohibited use pattern database.

Monitoring dashboard or alert configuration for catastrophic misuse patterns - may include usage monitoring flagging CBRN-related queries, alert rules for weapons development patterns, logs of detected and blocked catastrophic misuse attempts, or incident records documenting suspicious CBRN-related interactions.