The rapid expansion of artificial intelligence workloads has triggered a severe shortage in the global storage market. This crisis is disproportionately affecting non-profit organizations dedicated to preserving digital history. Entities like the Internet Archive and the Wikimedia Foundation report that skyrocketing prices for hard drives and memory are threatening their ability to maintain their vast archives.
Large-capacity drives see 3x price surge as hyperscalers book AI capacity.
Large-capacity hard disk drives, specifically those ranging from 28 to 30 terabytes, have seen costs increase by up to three times their previous levels. This price surge stems from shriveled production capacities that hyperscale cloud providers have largely booked for their own AI data centers. The scarcity extends to memory modules, which are also facing significant supply constraints.
The Internet Archive currently stores approximately 210 petabytes of data and adds 100 terabytes every day. Founder Brewster Kahle stated that maintaining this infrastructure has become a very real issue costing the organization time and money. A Wikimedia Foundation spokesperson confirmed similar struggles, noting that the primary impact is visible in the purchase of memory and hard drives, as well as in lead times for server deliveries.
Beyond hardware costs, these organizations face technical barriers from website operators. Stricter anti-scraping measures are now blocking the archival bots used by the Wayback Machine. These automated systems are increasingly being treated similarly to scrapers used for training artificial intelligence models, further complicating efforts to preserve the open web.
The storage crisis highlights the conflict between commercial AI infrastructure demands and public digital preservation. As hyperscalers prioritize their own capacity needs, non-profit archives are left to navigate a market defined by extreme price volatility and supply chain bottlenecks.
Discussion
0 comments
Log in to join the thread with a thoughtful take, question, or correction.