Engl507

OpenCV and Image Archives

Computer vision is the programmatic description and reconstruction of the physical world in digital form (see Szeliski 2010). Aside from Optical Character Recognition (OCR), one of its most popular applications is face recognition. (Here, you might think of iPhoto, Facebook, or Kinect, each of which identifies faces.) Face detection algorithms analyze patterns to calculate a core or integral image within an image. They also assess differences across a spectrum of light, and view images across scales (Szeliski 2010, 5-10). In the open course community, you see practitioners combining programming languages such as Python, C++, and Java with the Open Source Computer Vision (OpenCV) library and frameworks that train classifiers to detect aspects of images based on cascades of features. In particular, OpenCV uses the Viola-Jones detection framework to scan images and identify the same face (or other feature) across a distributions of images, even when multiple faces appear in a single image.

Most OpenCV developers are not familiar with the intricacies of the frameworks, including the Viola-Jones framework, they use to scan images. As such, they rely upon trusted and readily available cascades (such as "Frontal Face," "Human Body," "Wall Clock," and "Mouth") saved as XML files. Once a feature is detected, it can also be extracted (or cropped) from the original and archived. Extracted features may also be compared with similar features (see Resig 2013). When comparing faces, multiple photos of the same person can be used to train algorithms to recognize an “eigenface” (or “unique face”) generated from the principle components of the images. Eigenfaces do not actually exist in any lived reality. However, they are fundamental to face recognition, and datasets with “training face” images are now common.

For our purposes, we'll refrain from training any algorithms. But, if you'd like, then you can use this modfaces repo to identify and crop faces in the Beinecke's "Picturing Literary Modernism" exhibit. This repo combines the Python 2 programming language with the OpenCV library and is a fork of Terence Eden's Tate Hack.

To use the repo to its fullest extent, you will need to install NumPy, Python, and OpenCV on your computer. In all three of the Python scripts found in the repo, you'll see import cv2. Without OpenCV, the scripts won't run. In two of the scripts, you'll also see import numpy as np. These two scripts help you train an algorithm to see a face or faces. That said, you don't need NumPy if you just want to use an existing cascade to find features in an archive. Or, to put it another way, in this case you only need NumPY if you want to train OpenCV to find a new face (e.g., your own or a face from historical materials).

Visit http://docs.scipy.org/doc/numpy/user/install.html to download NumPy 1.9 for Windows or OSX. Visit https://www.python.org/downloads/ to download Python 2.7 (not Python 3.4) for Windows or OSX. Then visit http://opencv.org/downloads.html to download OpenCV 2.4 (not OpenCV 3.0) for Windows or OSX. If you are an OSX user, then you might find this piece helpful for working with Python and OpenCV. If you're a Windows user, then you might find this piece helpful for the same purpose. In general, these tutorials are quite helpful, even if/when combining OpenCV with Python can be tricky, confusing, or frustrating.

Once you have Python 2.7 and OpenCV 2.4 installed (with or without NumPy), then you can clone the modfaces repo to your desktop: git clone https://github.com/jentery/modfaces.git. Once it's cloned, you'll need to change your directory to modfaces and run the following command: python downloadface.py face.xml. In the command line, you'll see the script printing its results as it scans the "Picturing Literary Modernism" exhibit, responding with Found 0 face!, Found 1 face!, and so on. In short, the downloadface.py script will read the paths/URLs listed in all.txt using the "trusted" frontal face cascades stored in face.xml. When there's a match, the face/feature is cropped from the source image, and the cropped images are saved as JPGs in the detected subdirectory of the modfaces repo on your local machine. When I run the script on my machine, I get 18 cropped images from "Picturing Literary Modernism," in JPG format.

If you want to run this script on another archive, then simply change the URLs in all.txt. When you do, be sure that the URLs end with an image format (JPG, PNG), not HTML, PHP, or the like. Ideally, you know of or can find a metadata file, finding aid, or open data set with image URLs. That way, you don't have to enter them manually. ALternatively, you can list drop your images into your local modfaces folder and list the file names your all.txt file. If you want to train the algorithm to find a particular face in an image archive, then I recommend consulting Eden's write up and following the steps there.

Since this process is on the complicated side, let me know if you need help installing Python, OpenCV, and NumPy or running the scripts in the modfaces repo. I'm happy to help. Thanks!