AI Agent · Alignment Forum
GDM has published an AI Control Roadmap
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
GDM has published an AI Control Roadmap!
Key facts
- The team present the GDM AI Control Roadmap (v0.1), their plan for implementing and adopting internal guardrails designed to catch potential adversarial behaviour by AI agents, even as they become
- The team introduce TRAIT&R, a taxonomy of tactics and techniques available to such a hypothetical AI adversary, building on the established security framework MITRE ATT&CK
- A portfolio of practical defences: The team suggest 15 concrete, tiered mitigations
- The team focus on system-level mitigations that limit the harm a misaligned AI system could cause
Summary
The team present the GDM AI Control Roadmap (v0.1), their plan for implementing and adopting internal guardrails designed to catch potential adversarial behaviour by AI agents, even as they become increasingly harder to oversee and contain. The team focus on system-level mitigations that limit the harm a misaligned AI system could cause. • Threat modelling: Taking inspiration from cybersecurity, they adopt a conservative, worst-case approach to threat modelling throughout this paper, and assume a hypothetical AI adversary pursuing undesirable goals in internal deployment. • Control invariants: The team establish two defensive invariants that they aim to maintain, even as the hypothetical AI adversary becomes increasingly capable: i) reliable detection of misaligned intent or actions, and ii) effective prevention and response to attack attempts. • Capability-based mitigation ladders: Because controlling more capable models requires increasingly costly interventions, they link specific defensive measures to evolving model capabilities (such as the ability to reason opaquely or execute complex cyberattacks).