Open Source · OpenAI · Google · GPT · Greg Brockman · MIT Technology Review
These companies are chasing the next big thing in LLMs
Compiled by KHAO Editorial — aggregated from 1 source + 2 references discovered via search. See llms.txt for citation guidance.
◌ Single Source
MIT Technology Review ’s What’s Next series looks across industries, trends, and technologies to give you a first look at the future.
Key facts
- They trained a diffusion model that matched the performance of GPT-2—an LLM that OpenAI built in 2019—but was 10 times faster
- Inception claims its latest model, Mercury 2, performs as well as some of OpenAI’s GPT-4 models, released in 2023, but again 10 times faster
- OpenAI is set to spend $50 billion on computing this year, according to the company’s president, Greg Brockman
- Its latest models can run on a Raspberry Pi, a low-powered hobbyist computer that costs $50
Summary
Way back in the summer of 2017, AI researchers at Google put out a paper called “Attention Is All You Need,” in which they described a new type of neural network called a transformer. Nine years on, transformers are the engines inside every major large language model on the market. A growing number of scientists and engineers are now asking what’s coming next. Enter a wave of startups hoping to push the boundaries of this boomtown technology.