Archive.org and the Laura and John Arnold Foundation have jointly announced a $1.9 million grant “to develop a search engine that will provide unprecedented access to its extensive collection of webpages, also known as the Wayback Machine.” As previously reported, Archive.org has already been working on extensive upgrades of its front end and back-end systems, and this latest grant should consummate the process.
The Laura and John Arnold Foundation, founded by John D. Arnold, billionaire former head of energy-focused hedge fund Centaurus Advisors, and his wife Laura, exists “to address our nation’s most pressing and persistent challenges using evidence-based, multi-disciplinary approaches.” Most of its grants (around 30 percent) go to educational projects, but a further 20 percent go towards “evidence-based policy and innovation,” and 16 percent to “venture development.” According to the press release on the grant, “the search engine will allow researchers, historians, and others to retrieve data and information from the billions of webpages and websites stored in the Wayback Machine and will ensure that there is a comprehensive, open record of the Internet that is accessible to all.”
Archive.org’s own blog comment on the grant declares that “when completed in 2017, the next generation Wayback Machine will have more and better webpages that are easier to find.” According to Archive.org, the Wayback Machine “currently offers access to 439+ billion Web captures including Web pages, video and images.” Archive.org’s declared goals with the project include: Highlighting the provenance of pages found in the Wayback Machine; Rewriting the Wayback Machine code; Optimizing the scope and quality of pages we crawl; Improving the playback of media-rich and interactive websites; Updating the user interface; Finding websites based on keywords; and Partnering with other services to repair broken links by pointing to the Wayback Machine.
“Each day, more than 600,000 people use the Wayback Machine to access digital records of significant events ranging from the Iraq War to NASA’s exploration of Mars,” states the Laura and John Arnold Foundation release. “However, users are currently limited in their ability to find content because the portal’s search technology is outdated. The Wayback Machine also lacks an effective cataloging system.” These days the internet is often cited as humanity’s collective memory. Thanks to this project, that memory could be closer to achieving total recall.