OpenAI · GPT · Sam Altman · Fortune Technology
OpenAI quietly boosts some of Astra’s evaluation metrics, and continues to change others post-launch
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
OpenAI has changed several evaluation benchmarks for its GPT-6 Astra model since first publishing a blog post announcement mid-afternoon on Sept. 3.
Key facts
- Claude Fable 5.1 goes from 56.6% to 58.1%, and Opus 5 goes from 54.5% to 56.4%
- The scores for Astra’s predecessor, GPT-5.6 Sol, also went down from 12.2% to 9.4%
- Similarly, GPT-5.6 Sol’s scores go from 83%, down to 80.5%, and back up to 83% today
- In the first snapshot (2:23 p.m. on Sept. 3), Anthropic’s Fable 5.1 model’s score is 87.8%
Summary
The changes occurred amid an unusual rollout of the blog post. When OpenAI’s X account tweeted out the blog post at 3:32 p.m., the link was not loading properly, returning an error message. It turns out OpenaAI published the blog shortly after 2pm but retracted it for reason the company said it could not disclose, but which it said were unrelated to the benchmark performance figures. (OpenAI first told them it was a bug in the content management system, and then an internet outage.) Upon republishing the blog, it had different evaluation metrics that seemed to favor Astra—and some figures have continued to change even since then.