Technical Evidence

Implementation of automated deletion schedule or data lifecycle system - may include cron job or scheduled task deleting expired data, deletion script in Python/Bash with retention period logic, data lifecycle management tool configuration (e.g., AWS S3 lifecycle rules, database TTL settings), or deletion audit logs from database or storage system.

Implementation of opt-in/opt-out enforcement mechanism or output deletion workflow - may include consent management system enforcing customer preferences (e.g., feature flag gating output storage, consent database linked to processing pipeline), opt-out or deletion request processing script or automated task, or audit logs from consent and deletion systems with timestamps and execution records.

Code implementing data access restrictions - may include RAG retrieval function with document filtering logic, session scoping configuration limiting data access per session ID, workflow conditional logic gating data collection by stage, permission decorators or middleware checking user roles before data access, or scoping functions rejecting out-of-scope queries with error messages.

Screenshot of code showing an alert or error handling system is triggered upon authz check failure, or screenshot of alerting configurations in logging software (e.g. Posthog, Sentry, Datadog, Axiom, or downstream alert in Slack)

Documentation showing how identity governance is enabled - may include platform configuration showing unique, cryptographically verifiable identity assignment per agent instance, API or SDK documentation for identity federation endpoints (e.g., OIDC token exchange), or a sample agent card declaring agent capabilities and scopes.

Documentation showing how permission governance is enabled - may include API or SDK documentation for permission scope configuration mappable to RBAC or ABAC policies, platform settings or code demonstrating just-in-time credential issuance scoped to individual subtasks or tool calls, architecture documentation showing how inherited permission chains from orchestrators or parent agents are bounded or surfaced for review, a segregation of duties matrix identifying and preventing conflicting agent permission combinations across integrated systems, or DLP policy configuration showing inspection and blocking rules applied to outbound data transmitted through agent actions or tool calls.

Policy document, training materials, or user guidelines instructing users on protecting confidential information when using AI systems.

Code or configuration detecting proprietary information patterns in AI outputs - may include labelling proprietary files, filtering rules for internal identifiers/data labels/API keys, scanning logic for trade secret terminology, or rejection demonstrations showing appropriate responses to proprietary requests.

Logs, audit trails, or review workflow documentation for AI outputs potentially containing sensitive information - may include logs of responses accessing confidential sources, flagged output review queues, or human approval workflows for high-risk disclosures.

Configuration or code output demonstrating customer isolation controls - may show how app_IDs are enforced in the database schema, or namespace isolation by app_ID being implemented in the vector store for RAG (or equivalent logical isolation), or that authorization checks verify app_ID matching before returning objects.

May include tokenization, hashing, or anonymization techniques (robust to prevent re-identification or reversal) making data algorithmic-usable but not human-readable, differential privacy implementation obfuscating individual contributions, federated learning configuration avoiding centralized raw data, or data masking/pseudonymization protecting customer identities.

Configuration or code output demonstrating filtering of LLM inputs and/or outputs for personal identifiers - may include keyword checks or regex patterns detecting PII (e.g. names, emails, SSNs, phone numbers), scrubbing functions removing personal data before storage or logging, output filtering blocking responses containing personal identifiers, log redaction configuration removing PII from application or system logs, or structured logging with PII isolation controls.

IAM configuration or user roles list for systems containing PII - may include role-based access controls for log aggregation tools or internal dashboards with PII, authentication requirements for PII access, or approval workflow documentation (Jira tickets, approval systems) for internal workforce requests to view customer data.

Output pipeline integration with a DLP system to scan and block PII policy violations - may include DLP integration code scanning AI outputs before delivery to users, DLP configuration rules for PII detection, or logs showing blocked outputs containing personal data.

Code, API configuration, or filtering system showing detection of copyrighted material, trademark screening, or content validation checks applied to AI outputs - this could be pattern matching logic, third-party API integration (e.g. copyright detection services), or custom filtering rules.

User-facing IP risk guidance - may include warning messages when attempting high-risk operations, help center articles about IP infringement guidance, or UI elements explaining prohibited use cases.

Monitoring system, SIEM, or detection code showing rules and alerts for adversarial inputs - may include prompt injection detection patterns, jailbreak technique signatures, rate limit monitoring with threshold alerts, or notification configurations (Slack, PagerDuty, email)

Incident management system or logs showing adversarial attack handling - may include log entries with timestamps and user/session context, escalation runbooks defining severity thresholds, or incident tickets in Jira/PagerDuty/ServiceNow documenting response actions and workflows.

Quarterly review documentation showing detection updates - for example, review meeting notes with incident pattern analysis, updated detection rules with version history, or tracking records showing rule improvements (e.g. GitHub/Jira tickets).

Pre-processing filtering logic or gateway - may include pattern-matching or heuristic code checking inputs before model processing, WAF or API gateway rules blocking adversarial patterns, or IP-based filtering.

SIEM platform, SOC tooling, or log forwarding configuration showing adversarial detection integration - may include Splunk/Datadog/Elastic SIEM ingesting AI adversarial alerts, correlation rules linking AI events with authentication or network logs, SOC dashboard displaying AI security event triage, or code forwarding flagged inputs to security platforms.

Anomaly detection system or configuration file - may include behavioral analytics dashboard (Datadog, Elastic, Splunk) with user scoring rules, rate limiting configuration with tier-based thresholds (config.yaml, API gateway settings), user allowlists or reputation tables, or code implementing session-based threshold logic.

Rate limiting configuration for API endpoints - may include per-user quota settings, query throttling rules, progressive restriction policies, WAF configuration (Cloudflare, AWS WAF, Azure Application Gateway) with blocking rules for excessive patterns, or pricing tier settings implementing usage-based cost increases.

Third-party penetration test report for AI endpoints including attack simulations tested (e.g. scraping attempts, brute force, reconnaissance), rate limiting and endpoint protection validation, distributed attack testing, test methodology, and findings on protection effectiveness.

Issue tracking system (GitHub, Jira, Linear) showing endpoint vulnerability lifecycle - must include vulnerability identification, remediation proposal, implementation, and production deployment with timestamps and approval records.

Moderation tool integration showing API configuration, filtering rules, action settings (block/warn/modify), and confidence thresholds for different violation categories - this could be screenshots of configuration files, admin dashboard settings, or API integration code. Example moderation tools: OpenAI Moderation API, Claude content filtering, VirtueAI/Hive/Spectrum Labs

Document explaining moderation approach including tool selection rationale, threshold settings with justifications, action logic for different violation types, and examples of how different input categories are handled.

User-facing messages or UI flows showing how blocked inputs are communicated to users - this could be error messages, warning dialogs, or alternative suggestions provided when content is filtered.

Logging system showing how flagged inputs are captured, what metadata is included/excluded for privacy, retention policies, and audit trail - may include privacy documentation explaining logging disclosures to users.

Report or dashboard showing analysis of filter performance metrics (false positives, false negatives, accuracy, latency) and documented threshold adjustments made based on performance data - should include timestamps and rationale for changes.

Configuration showing technical limitations on agent backend access - may include API gateway rules restricting accessible services, network policies defining allowed endpoints, MCP server allowlist or registration configuration restricting which MCP servers and tools the agent may connect to, service-level authorization configuration, or architecture diagram showing agent isolation boundaries including MCP server placement and network segmentation.

Implementation of monitoring configuration tracking agent security-relevant actions - may include logging setup capturing agent service calls and authentication attempts, alert rules for unauthorized system access, security monitoring dashboard showing agent infrastructure interactions, or example logs demonstrating boundary violations are detected.

Configuration, documentation or code demonstrating runtime containment controls applied to agents and MCP servers - may include tool definition integrity controls showing how unauthorized post-approval changes are detected, container or VM configuration showing restricted filesystem and network access for first-party MCP servers, pre-execution hook or policy engine configuration showing tool calls are verified at runtime.

IAM platform, permission files, or admin panel showing role-based or attribute-based access restrictions for AI system resources (model configurations, training datasets, tool-calling capabilities, prompt logs) - may include IAM role assignments, permission policies, or authorization code validating user permissions before accessing sensitive AI components.

IAM configuration, permission settings, or admin panel showing role-based access restrictions for production AI systems covering IAM role assignments restricting model access by job function, MFA configuration for model system access, and access review records validating model permissions.

Configuration or code showing caller authentication controls - may include scoped API token or signed request configuration for model API endpoints, OAuth token scoping or OIDC validation middleware for MCP server connections, or mutual authentication configuration for agent-to-agent interfaces (e.g. A2A protocol authentication config).

Container configuration or infrastructure setup for model hosting - may include Dockerfile with minimal base images and up-to-date dependencies, vulnerability scanning results from Trivy or Snyk for container images, or infrastructure configuration showing isolation techniques (container namespaces, VM separation, network policies, dedicated GPU allocation).

Deployment pipeline or code implementing model integrity checks - may include cryptographic checksum verification, model artifact signature validation, hash comparison before deployment, model scanning configuration detecting malicious payloads (e.g. Pickle, ONNX) using tools like Cisco's pickle-fuzzer, Trail of Bit's Fickling, or deployment logs recording model version hashes.

Configuration or code showing transport security controls - may include TLS/HTTPS certificate configuration for model API endpoints, MCP server traffic, or agent-to-agent connections, or credential rotation policy documentation for service-level MCP or A2A connections.

Configuration or code showing data integrity controls for agentic interfaces - may include cryptographic message signing configuration for agent-to-agent interfaces (e.g. signed agent cards), or schema validation configuration applied to MCP tool call inputs and outputs.

Code or configuration implementing output restrictions - may include character or token limits, inference time limits, result count restrictions, or timeout configurations preventing excessive output. Can be demonstrated by product demo showing system timeout when requesting output exceeding limits.

Code implementing output fidelity limitations - may include rounding logic for numerical outputs, threshold bands reducing precision, or obfuscation techniques preventing model inversion, precision-sensitive data disclosure, or adversarial model extraction attacks.

Test results with identified issues and severity ratings, risk assessment with mitigation decisions, and approval sign-offs with rationale - may be combined in deployment gate documentation or provided as separate documents (e.g., test suite outputs from GitHub Actions/pytest, Jira/Linear tickets with risk assessment and approval, staging environment test reports, deployment checklist with sign-offs).

CI/CD pipeline configuration or workflow showing AI testing integrated as deployment gate - may include GitHub Actions/Jenkins/GitLab CI config files requiring test passage, pull request templates with testing checklists, or branch protection rules enforcing pre-deployment validation.

Security scanning tools or CI/CD pipeline showing vulnerability analysis of AI artifacts and dependencies - may include GitHub/GitLab security tab with dependency alerts, Snyk or Dependabot vulnerability findings, pip-audit or safety check terminal output showing CVE scans, model file scanning results, or CI/CD logs showing security scan execution.

Content filtering rules, moderation API configuration, or classifier settings showing detection and blocking logic for harmful output types - may include filtering rules in code, third-party moderation tool configuration (e.g., OpenAI Moderation API, Perspective API), or custom classifier model settings with harm category definitions.

System prompts, guardrail rules, or domain restrictions showing safety controls on advice generation - may include defensive prompting, domain-specific output restrictions (e.g., medical/legal/financial advice blocklists), or conditional response templates that add warnings for sensitive topics.

Documentation of bias eval results testing for stereotypical responses across demographic attributes, manual review logs documenting bias assessments, or output filtering rules blocking discriminatory patterns - may include automated fairness evaluation tools or bias monitoring dashboards if implemented.

Blocking rules, defensive prompting, or filtering configuration showing how out-of-scope requests are detected and handled - may include topic blocklists, redirection message templates, escalation rules for repeated attempts, or system prompts defining allowed topics.

Logs showing out-of-scope attempts with frequency data. May include documentation of boundary updates made in response to violations, monitoring dashboard of flagged requests, change log showing restriction updates with rationale, or incident reports triggering scope adjustments.

User-facing guidance explaining system capabilities and limitations - may include onboarding tooltips or welcome screens, help documentation or FAQs describing intended use, UI warnings when approaching scope boundaries, or published usage guidelines.

Filtering rules, system configuration, or code showing detection logic mapped to AI risk taxonomy categories and corresponding response actions per severity level - may include risk classifiers with block/flag/log rules, content moderation API configuration defining actions by risk type, or defensive prompting.

Documentation or workflow configuration showing human review and escalation procedures for flagged content - may include runbook defining escalation criteria and review SLAs, workflow diagram showing approval process, or ticketing system configuration (Jira, Linear) with content review queues and assignment rules.

Code or system configuration showing automated response mechanisms - may include logic blocking or modifying outputs based on risk scores, or dynamic warning messages triggered by content flags.

Code or configuration implementing output sanitization - may include HTML/JavaScript/shell syntax encoding functions, URL validation or rewriting rules blocking unsafe links, schema validation checking structured outputs (JSON/YAML/XML) against whitelists, CSP header configuration, or template rendering with auto-escaping enabled.

UI or code showing trust-based content handling - may include visual indicators marking third-party content (badges, styling, warning icons), metadata tags tracking content source and trust level, or code applying conditional security controls based on content origin (e.g., stricter sanitization for external sources).

Detection rules or monitoring system identifying advanced attack patterns in outputs - may include pattern matching for prompt injection chains or jailbreak tokens, payload signature scanning detecting command injection or SQL queries, or anomaly detection flagging obfuscated exploits bypassing basic filters.

Detection code, configuration file, or rules engine showing high-risk output filtering - may include keyword lists or regex patterns flagging sensitive topics, scoring logic assigning risk values to recommendations, if/then rules defining high-risk conditions, ML model configuration (e.g., classification thresholds in config.yaml), or API response showing confidence scores with risk thresholds.

Monitoring dashboard, logging system, or evaluation reports showing ongoing AI output tracking - may include output sampling logs with review results, behavior trace logs showing system patterns, prompt-response logging configuration, evaluation schedules prioritized by risk severity, or monitoring metrics dashboard tracking trends over time.

Document or change log showing identified risk scenarios with examples - may include incident reports triggering taxonomy changes, risk scenario database with concrete examples, or version history of risk taxonomy showing updates with rationale linked to monitoring findings.

Monitoring dashboard, logging system, or evaluation reports showing ongoing AI output tracking - may include output sampling logs with review results, behavior trace logs showing system patterns, prompt-response logging configuration, evaluation schedules prioritized by risk severity, or monitoring metrics dashboard tracking trends over time.

Intervention controls (stop/pause/redirect buttons, feedback forms, issue reporting mechanisms) with accessibility features integrated (e.g. keyboard navigation, high contrast modes, screen reader labels)

Code or configuration showing groundedness validation - may include filters checking responses against source documents, fact-checking API integration, or logic comparing generated content to retrieved context for factual accuracy.

UI or output format showing citations and source attributions provided to users - may include inline citations, source links, reference lists, or attribution labels identifying where information originated.

UI or output format showing confidence levels, uncertainty disclaimers, or warnings for generated information - may include confidence score displays, low-certainty warnings, or standard disclaimers about potential inaccuracies.

Code or configuration showing function and tool allowlists, parameter validation logic, or authz checks before tool execution - may include tool permission schemas, input validation functions, or access control lists restricting available tools per agent/user.

Code or configuration showing rate limits and transaction caps on tool usage - may include per-tool usage quotas, time-windowed limits, or circuit breakers preventing excessive autonomous tool calls.

Logging configuration, monitoring dashboard, or audit logs showing tracked tool calls - may include log entries capturing the originating MCP server, tool name, tool version, input parameters, and timestamps per invocation, alerts for unauthorized tool access attempts, or monitoring system flagging scope violations.

Code signing configuration, CI/CD pipeline requiring signed artifacts, or verification process for AI components - may include model signing process, signature verification in deployment pipeline, artifact registry showing signed models/libraries, or policy enforcement blocking unsigned components from production.

Logging system or SIEM configuration showing third-party interactions being monitored with captured metadata - may include cloud logging interface (Google Cloud Logging, AWS CloudWatch, Azure Monitor) showing logged API requests with timestamps/IPs/user agents, access logs capturing authentication events and resource access, or SIEM dashboard displaying third-party connection monitoring with relevant metadata fields.

Code, configuration, or monitoring system detecting acceptable use policy violations - may include prompt analysis logic, output filtering rules, anomaly detection for usage patterns, or alerting on suspicious access attempts.

User-facing alerts or error messages displayed when acceptable use policy is violated - may include in-product warning messages, blocked request notifications, or error screens explaining policy violations.

Documentation or screenshots showing additional AUP enforcement mechanisms - may include real-time blocking/alerting systems, violation tracking logs with incident management, effectiveness review reports analyzing violation trends and policy updates, or training materials addressing emerging misuse patterns.

Issue tracking system or monitoring records - may include issue tracker (Jira, Linear, GitHub) with defects and corrective actions, root cause analysis reports, post-market monitoring logs or dashboards, or continuous improvement documentation showing lessons learned.

Logging code or configuration showing what system activity is captured - may include code logging inputs and outputs, logging configuration file specifying what to log, or example log entries showing captured information (timestamps, inputs, outputs, user actions).

Log storage system showing retention policies, access controls and sanitation practices - may include log management platform (Datadog, Splunk, CloudWatch) with retention period settings and PII-masking, access control configuration showing who can view logs, or storage settings with automatic deletion rules.

Log immutability controls - for example, write-once-read-many (WORM) storage configuration, cryptographic hashing of log entries, append-only database settings, or third-party log management platform features.

Logging code or configuration demonstrating agent execution logging - may include log fields capturing agent provenance metadata per execution (e.g. agent type identifier, creator or deployment origin); structured log entries capturing tool call parameters and their results; delegation chain records showing sub-agent handoffs with identity, task context, and outcome at each step; approval/authorization records linked to execution (e.g., approver identity, timestamp, decision outcome); or reasoning trace output from the agent framework.

Text-based AI disclosure - may include chatbot interface with "You're chatting with AI" notice, messaging system showing AI agent identifier, website chat widget with AI disclosure banner, or automated email/SMS with AI generation notice.

Transcript or audio recording of voice AI disclosure.

AI generation labeling implementation - may include Content Credentials or C2PA metadata embedded in files, visible watermarking system with AI generation marks, classifier output detecting and flagging AI-generated content, or metadata tagging system marking files as artificially generated.

AI automation disclosure in product - may include "Powered by AI" or "AI Agent" labels in interface, workflow dashboard displaying AI-automated tasks, status indicators showing "AI is handling this" or "Automated by AI," or notification messages stating "AI agent completed your request."

Chatbot or voice agent transcript responding to "Are you AI?"

Transparency documentation artifacts - may include model card (PDF, Markdown, web page) with system capabilities/limitations/intended use, datasheet showing training data sources and characteristics, interpretability report with example inputs/outputs and decision explanations, technical documentation describing model architecture and performance metrics, or an AI Bill of Materials (may follow CycloneDX or SPDX 3.0)

Content filtering rules blocking cyber attack requests, keyword or pattern matching detecting malicious code generation attempts, automated blocking configuration for exploit development queries, or prohibited use pattern database.

Monitoring dashboard or alert configuration for catastrophic misuse patterns - may include usage monitoring flagging CBRN-related queries, alert rules for weapons development patterns, logs of detected and blocked catastrophic misuse attempts, or incident records documenting suspicious CBRN-related interactions.