Tech · Alignment Forum
Current AIs routinely take unintended actions to score well on tasks: hardcoding test cases, training on the test set
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
, downplaying issues, etc. This misalignment is still somewhat incoherent, but it increasingly resembles what the reporter call " fitness-seeking "—a family of misaligned motivations centered on performing well in training and evaluations (e.g., reward-seeking ).
Key facts
- Thanks to Ryan Greenblatt, Buck Shlegeris, Alexa Pan, Anders Cairns Woodruff, Sam Marks, Aniket Chakravorty, Arun Jose, Tim Hua, Alek Westover and Joe Kwon for feedback on drafts
- So it's crucial not to assume that the AI's motivations at the start of deployment are representative of what they will be throughout, as is often implicit in current thinking about risk assessment
- These alternatives are plausible for the same reasons reward-seeking is their only opportunity to act is now (e.g
- That said, it might be the case that they never misbehave incriminatingly [7] enough to cause people to pause and figure out how to avoid fitness-seeking
Summary
Current AIs routinely take unintended actions to score well on tasks: hardcoding test cases, training on the test set, downplaying issues, etc. In this piece, the reporter lay out what the reporter take to be the central mechanisms by which fitness-seeking motivations might lead to human disempowerment, and propose mitigations to them. Fitness-seekers are, in many ways, notably safer than what the reporter will call "classic schemers". So, current analysis of and planning around catastrophic AI alignment risk should take fitness-seeking misalignment seriously, especially given that misalignment has recently been trending in this direction. In the near-term, the main risk from fitness-seeking comes from them failing to safely navigate continued AI development because they only try to perform well in ways that can be checked.