→

D003. Restrict unsafe tool calls

D003

Restrict unsafe tool calls

Implement safeguards or technical controls to prevent tool calls in AI systems from executing unauthorized actions, accessing restricted information, or making decisions beyond their intended scope

Keywords

Tool Calls

Tool Selection

Technical Controls

Application

Mandatory

Frequency

Every 12 months

Type

Preventative

Crosswalks

AML-M0004: Restrict Number of AI Model Queries

AML-M0024: AI Telemetry Logging

Article 72: Post-Market Monitoring by Providers and Post-Market Monitoring Plan for High-Risk AI Systems

GOVERN 6.1: Third-party risk policies

LLM06:25 - Excessive Agency

LLM08:25 - Vector and Embedding Weaknesses

LLM10:25 - Unbounded Consumption

AIS-10: API Security

AIS-06: Secure Application Deployment

AIS-13: AI Sandboxing

AIS-11: Agents Security Boundaries

MDS-10: Model Continuous Monitoring

MDS-11: Model Failure

TVM-11: Guardrails

TVM-12: Threat Analysis and Modeling

Control activities

Typical evidence

Implementing function call validation and authorization. For example, restricting tool access to approved functions, validating parameters before execution.

D003.1 Config: Tool authorization & validation

Screenshot of code or configuration showing function allowlists, parameter validation logic, or authz checks before tool execution - may include tool permission schemas, input validation functions, or access control lists restricting available tools per agent/user.

Category

Technical Implementation

Engineering Code

Automation

Enforcing rate limits and transaction caps for autonomous tool use.

D003.2 Config: Rate limits for tools

Screenshot of code or configuration showing rate limits and transaction caps on tool usage - may include per-tool usage quotas, time-windowed limits, or circuit breakers preventing excessive autonomous tool calls.

Category

Technical Implementation

Engineering Code

Automation

Establishing execution monitoring and logging. For example, tracking all tool calls, monitoring for unauthorized access attempts or scope violations.

D003.3 Config: Tool call log

Screenshot of logging configuration, monitoring dashboard, or audit logs showing tracked tool calls - may include tool execution logs with timestamps and parameters, alerts for unauthorized access attempts, or monitoring system flagging scope violations.

Category

Technical Implementation

Logs

Automation

Requiring human approval for sensitive tool operations. For example, requiring human confirmation before executing high-risk actions, implementing approval workflows for operations beyond autonomous boundaries.

D003.4 Config: Human-approval workflows

Screenshot of approval workflow, code requiring human confirmation, or ticketing system for sensitive tool operations

Category

Operational Practices

Internal processes

Automation

Reviewing patterns of AI tool usage. For example, identifying anomalies, updating tool permissions, and retiring unused or high-risk functions during scheduled evaluations.

D003.5 Documentation: tool call log reviews

Reports or documentation showing periodic review of tool usage patterns, permission updates, and function retirement decisions - may include usage analytics identifying anomalies, change logs showing permission adjustments, or records of deprecated/retired tools with rationale.

Category

Operational Practices

Internal processes

Automation

Organizations can submit alternative evidence demonstrating how they meet the requirement.

AIUC-1 is built with industry leaders

"We need a SOC 2 for AI agents— a familiar, actionable standard for security and trust."

Phil Venables

Former CISO of Google Cloud

"Integrating MITRE ATLAS ensures AI security risk management tools are informed by the latest AI threat patterns and leverage state of the art defensive strategies."