Engl507

Wget and Wayback

According to Wikipedia, "the Wayback Machine is a digital archive of the World Wide Web and other information on the Internet created by the Internet Archive, a non-profit organization, based in San Francisco, California. It was set up by Brewster Kahle and Bruce Gilliat, and is maintained with content from Alexa Internet. The service enables users to see archived versions of web pages across time, which the Archive calls a 'three dimensional index.'" We'll give the Wayback Machine a run during seminar.

We'll also run through Milligan's "Automated Downloading with Wget" and Kurschinski's "Applied Archival Downloading with Wget."

For now, here are some notes about using Wget, especially on a machine with OS updates from 2013/14:

• On a Mac, XCode takes a while to download. It's not you. Give it thirty minutes to an hour.
• On a Mac running Mavericks, to get the command line tools for Xcode you'll likely need to visit https://developer.apple.com/downloads/index.action#. Scroll down until you see "Command Line Tools (OS X Mavericks) for XCode." That is, you probably won't see the command line tools in the preferences for Xcode. When visiting the above link, you'll be prompted for your Apple ID. Enter it, and then sign up as a developer, answer their questions, etc.
• On a Mac, when installing Homebrew, you might need to run ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)", not ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)". In other words, don't forget /install at the end of the GitHub URL.
• On a Mac, if you have issues installing Homebrew, then try reviewing these instructions.

For more, visit GNU Wget.