FT.com [paywall]: The Internet Archive’s mission is to ‘provide universal access to all knowledge’. Within the partitions of a lovely former church in San Francisco’s Richmond district, racks of laptop servers hum and blink with exercise. They comprise the web. Well, a really great amount of it. The Internet Archive, a non-profit, has been gathering net pages since 1996 for its famed and beloved Wayback Machine. In 1997, the gathering amounted to 2 terabytes of knowledge. Colossal again then, you possibly can match it on a $50 thumb drive now. Today, the archive’s founder Brewster Kahle tells me, the venture is getting ready to surpassing 100 petabytes – roughly 50,000 instances bigger than in 1997. It comprises greater than 700bn net pages. The work isn’t getting any simpler. Websites at the moment are extremely dynamic, altering with each refresh. Walled gardens like Facebook are a source of frustration to Kahle, who worries that a lot of the political exercise that has taken place on the platform could possibly be misplaced to historical past if not correctly captured. In the title of privateness and safety, Facebook (and others) make scraping tough. News organisations’ paywalls (such as FT’s) are additionally “problematic”, Kahle says. News archiving was taken extraordinarily significantly, however adjustments in possession and even only a web site redesign can imply disappearing content material…”

See also LLRX – Fenced-off culture, the privatized Internet, and why book publishers lean on a 30-year-old doctrine