AI Agent · MIT Technology Review
How “reverse game theory” could keep AI agents in check
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
A new approach known as “reverse game theory” aims to create incentives for powerful AI models that make following the rules the most rational choice.
Key facts
- An exclusive interview with Nick Cammarata, the researcher whose 26,000-word viral blog post has AI safety folks celebrating
- Hazan points to Nobel Prize–winning work by the economist Gary Becker, who introduced the idea of the “rational offender” in the 1960s
- Ten years after AlphaGo’s match against Go champion Lee Sedol, today’s AI still isn’t tapping into the machinery that made that win possible
- AI agents have a good understanding of what other AI agents can achieve,” Morris says
Summary
As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law, hack into unauthorized servers, or release poison gas into major cities and kill them all. Until now, this work has mostly focused on trying to peer into an AI’s “brain” to understand what it is “thinking,” or training models to hew to values that they hold dear. The trouble is, these techniques don’t seem to work particularly well: Models have been shown to deliberately underperform on tests, hide their intentions, and mislead their evaluators. Now some alignment researchers believe that reverse game theory, also known as mechanism design, might get better results.