OpenAI · Elon Musk · MIT Technology Review
The emergence of the web data infrastructure layer for AI
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
In partnership with Bright Data.
Key facts
- Some research has found that 97% of AI organizations depend on real-time web data infrastructure, but 90% feel boxed in by various restrictions
- As Lenchner explains, “It's having infrastructure that can mimic a web user with identifying information—IP address, location, and 1,000 more parameters
- To learn more from Bright Data, read the Data for AI 2026 report
- In fact, one survey found that 56% of AI practitioners said businesses need access to real-time web data to improve trust in AI outputs
Summary
New use cases are emerging each day. To understand this challenge, consider the foundation of the web itself. The next frontier in AI may depend on a new web data infrastructure layer that can enable models to discover and map this ever-expanding digital realm. “The data suggests there's far more data out there,” says Or Lenchner, CEO of Bright Data, a web data collection platform. While early AI breakthroughs were driven by scaling training data and model size, organizations are now encountering a fundamental bottleneck: They need to keep pace with the dynamic, unstructured, and constantly evolving nature of web data to ground outputs in current and verifiable information.