Anthropic · Gemini · Claude · OpenAI · ChatGPT · MIT Technology Review
The AI Observatory flagged that AI deploy differs significantly across models and has changed over time
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
The Anthropic Economic Index is one of the best-known and most widely cited sources of AI usage data, but it has blind spots.
Key facts
- To create the AI Observatory, Reuel and researchers from MIT, Stanford, the Data Provenance Initiative, and other institutions aggregated 85,633 conversational turns (that is, the user prompt
- The datasets the AI Observatory looked at include conversations that took place between 2023 and 2025, and it found differences both in how people were using AI and how various AI platforms responded
- There is no independent source to corroborate it,” says Anka Reuel, a computer science PhD candidate at the Stanford Trustworthy AI Research (STAIR) Lab
- When the AI Observatory researchers applied Anthropic’s methods to their dataset, they found that nearly half the conversations—48%—would have been filtered out
Summary
AI companies like Anthropic and OpenAI regularly publish reports on how people are using products like Claude and ChatGPT, but they only release the data they want them to see, AI researchers say. “There is no independent source to corroborate it,” says Anka Reuel, a computer science PhD candidate at the Stanford Trustworthy AI Research (STAIR) Lab. Reuel is co-lead of a new research project, called the AI Observatory, that aims to fill the gap. The AI Observatory found that AI use differs significantly across models and has changed over time.