In addition to working with a single image file, it is also a common task to work with a large number of image files. Rather than loading them individually, the ImageSet() library manages sets of images. The following example demonstrates how to use an ImageSet to manage the saving of a set of files from a security camera. The ImageSet() command takes one optional parameter: the path to the directory containing the images to be loaded. If a directory argument is not provided, ImageSet will create an empty list to which images can be added.

from SimpleCV import Camera, ImageSet
import time

cam = Camera()

camImages = ImageSet()  

# Set to a maximum of 10 images saved
# Feel free to increase, but beware of running out of space
maxImages = 10

for counter in range(maxImages):
     # Capture a new image and add to set
     img = cam.getImage()

     camImages.append(img)  

     # Show the image and wait before capturing another
     img.show()

     time.sleep(6)

camImages.save(verbose=True)  

As a final note on ImageSets; like an Image, an ImageSet also has a show() function. Where the show() function displays a single image when called on an Image object, it will show a slideshow when called on an ImageSet object. In the case of ImageSets, the show function takes one argument that sets the number of seconds to pause between images. For example, ImageSet().show(5) will display a slide show with a five=-second delay between showing each image in the set.

The Open Kinect project provides provides free drivers that are required to use the Kinect. The standard installation on both Mac and Linux includes the Freenect drivers, so no additional installation should be required. For Windows users, however, additional drivers must be installed. Because the the installation requirements from Open Kinect may change, please see their website for installation requirements at http://openkinect.org.

As mentioned above, the Kinect is a combination of a standard 2D camera with a second depth sensor to capture 3D information. The overall structure of working with the 2D camera is similar to a local camera. However, initializing the camera is slightly different:

from SimpleCV import Kinect

# Initialize the Kinect
kin = Kinect()  

# Snap a picture with the Kinect
img = kin.getImage()  

img.show()

Using the Kinect simply as a standard 2D camera is a pretty big waste of money. The Kinect is a great tool for capturing basic depth information about an object. Underneath the hood, it measures depth as a number between 0 and 1023, with 0 being the closest to the camera and 1023 being the farthest away. Although the Kinect captures values in a range from 0 to 1023, the SimpleCV framework automatically scales that range down to a 0 to 255 range. Why? Instead of treating the depth map as an array of numbers, it is often desirable to display it as a grayscale image. In this visualization, nearby objects will appear as dark grays, whereas objects in the distance will be light gray or white. To better understand this, the following example demonstrates how to extract depth information:

from SimpleCV import Kinect

# Initialize the Kinect
kin = Kinect()

# This works like getImage, but returns depth information
depth = kin.getDepth()

depth.show()

The example output, shown in Figure 3-2, shows some hints of a person in the foreground, as indicated by the darker person-shaped spot. Other, more distant objects are also somewhat discernible further in the background. The image is not a traditional picture, but the relative distance of the objects still provides some indication or outline of the actual objects.

Figure 3-2. A depth image from the Kinect

The Kinect’s depth map is scaled so that it can fit into a 0 to 255 grayscale image. This reduces the granularity of the depth map. If needed, however, it is possible to get the original 0 to 1023 range depth map. The function getDepthMatrix() returns a NumPy matrix with the original full range of depth values. This matrix represents the 2×2 grid of each pixels depth. More information about the link between matrices and images is covered in Chapter 4.

from SimpleCV import Kinect

# Initialize the Kinect
kin = Kinect()

# This returns the 0 to 1023 range depth map
depthMatrix = kin.getDepthMatrix()

print depthMatrix

Putting the pieces together, it is possible to create a real-time depth camera video feed using the Kinect. These examples are best run as a separate Python script, rather than in the SimpleCV shell. The three examples are:

The first example is a basic streaming feed from the Kinect. Like the examples in Chapter 2 that used the webcam to create a video feed, this provides a real-time stream of images. Unlike the previous examples, however, it shows the depth map instead of the actual image.

from SimpleCV import Kinect

# Initialize the Kinect
kin = Kinect()  

# Initialize the display
display = kin.getDepth().show()

# Run in a continuous loop forever
while (True):
     # Snaps a picture, and returns the grayscale depth map
     depth = kin.getDepth()

     # Show the actual image on the screen
     depth.save(display)

The classic real world application for an IP camera is a security camera. A WiFi connected web camera can easily stream a live video feed to a central location for monitoring. These examples are focused on basic image capture. Later chapters of the book will talk about how to detect motion and other ideas for creating a more robust application. The two versions of the security camera application are demonstrated:

As with the previous examples, these are best run as Python scripts—and not in the SimpleCV shell.

The first example is basic single IP camera. Once configured, it works like a locally connected camera. As a demonstration, the following example shows a feed captured from the IP camera:

from SimpleCV import JpegStreamCamera, Display
import time

#initialize the IP camera
cam = JpegStreamCamera("http://35.13.176.227/video.mjpg")  

display = Display()  

img = cam.getImage()
img.save(display)

while not display.isDone():

     img = cam.getImage()
     img.drawText(time.ctime())
     img.save(display)

     # This might be a good spot to also save to disk
     # But watch out for filling up the hard drive

     time.sleep(1)

One approach to using existing images is to use a virtual camera. Instead of capturing data fed through a camera, the virtual camera loads a video file that is accessed as though it is a stream of video coming through a camera. By this point in the book, the overall access and use should appear familiar to most readers:

from SimpleCV import VirtualCamera

# Load an existing video into the virtual camera
vir = VirtualCamera("chicago.mp4", "video")

vir.getImage().show()

The previous example looks for a video named chicago.mp4 as the frame source. The first parameter to VirtualCamera() is the filename of the video to load. The second is simply the word “video,” indicating that the first parameter points to a video as opposed to a static image. When working with a virtual camera, each call to getImage() will advance the video by a single frame. Figure 3-3 shows one frame from the example video. Once the last frame of the video is reached, calling getImage() again will loop back to the first frame of the video.

Figure 3-3. Example output from the example video file, opened with the virtual camera

Because virtual cameras are based on the Camera class, other camera functionality also works. For example, the Camera.live() function introduced in the previous chapter will work with virtual devices too. Just like with a regular webcam, simply click with the left mouse button on any point on the screen to get information about the pixel coordinates and color. Notice that at the end of the video, it automatically loops back to the beginning and plays it again.

from SimpleCV import VirtualCamera

vir = VirtualCamera("chicago.mp4", "video")

# This plays the video
vir.live()

Figure 3-4 shows an example frame from what you would see using this code.

Figure 3-4. Example of the virtual feed using the live() function

Instead of a video file, a single image can also be used as a virtual camera:

from SimpleCV import VirtualCamera

# Notice: the second parameter is now: image
vir = VirtualCamera("chicago.png", "image")

vir.getImage().show()

This time, the VirtualCamera function is passed the path to an image file and then the word “image,” indicating that the first parameter is a single image. Because this is only a single image and not a video, getImage() always returns the same image. A lot of the same functionality could be achieved by simply loading the image. In fact, the following two lines of code create identical output to the end user:

from SimpleCV import Image, VirtualCamera

# These two lines of code do the same thing
VirtualCamera("chicago.png", "image").getImage().show()
Image("chicago.png").show()

Notice that the overall functionality of the virtual camera with single images looks a lot like working with the Image library. Because the Image library also includes additional features to handle drawing and the extraction of features, it is usually the preferred method of working with images.

This example uses the ImageSet library to convert an entire directory of images to the .jpg format. It first uses the ImageSet to load the directory of images. It then iterates through the set, changing the name of the file to have a .jpg extension. Then it saves the file again with the new file extension and automatically converts it to the new file format during the save process.

from SimpleCV import ImageSet

set = ImageSet(".")  

for img in set:  

     oldname = img.filename  

     newname = oldname[0:-3] + 'jpg'  

     print "Converting " + oldname + " to " + newname

     img.save(newname)  

For the next application, the depth information from the Kinect can be used to extract objects from the foreground and then erase the background. In computer vision, this is known as segmentation, which is the process of dividing an image into groups of related content in order to make the image easier to analyze. Segmentation is covered in greater depth in later chapters, but this example shows how the Kinect can also be used to perform basic segmentation on the image.

from SimpleCV import Kinect
import time

# Initialize the Kinect
kin = Kinect()

# Get the image and depth information
dep = kin.getDepth()  
img = kin.getImage()  

# Turn into a pure black and white image for segmentation
fore = dep.binarize(190).invert()  
fore_only = img - fore  

fore_only.show()  

# Keep the image open for 10 seconds
time.sleep(10)

The resulting segmentation is not perfect, as you can see in Figure 3-5. It includes a little stuff from right and above the object and misses some material from left and below the object. This is an artifact of the distance between the normal image camera and the depth sensor. Objects farther away from the camera will have less of a problem with this.

Figure 3-5. Example result of using depth for segmentation. From L–R: The 2D image Center; the depth information; the final result

This is the first of several tricks for extracting important features from an image. It demonstrates how the depth information can be used to reduce the image to only its key components. As the book progresses, we cover these concepts in greater detail, and introduce a variety of different tips and tricks.

The final Kinect example goes a little further. As demonstrated in the previous example, the 3D depth information is useful for identifying an object of interest because the object of interest is likely closer to the camera than the background objects. This example detects the largest object in the field of view and then tries to measure its height.

from SimpleCV import Kinect, Display
import time

# Initialize the Kinect
kin = Kinect()

# Initialize the display
disp = Display((640, 480))  

# This should be adjusted to set how many pixels
# represent an inch in the system's environment
pixelsToInches = 6  

while not disp.isDone():

     img = kin.getDepth()

     blobs = img.binarize().findBlobs()  

     if (blobs):
          img.drawText(str(blobs[-1].height()/ pixelsToInches) + " inches",
                    10, 10)  

     img.save(disp)

     time.sleep(1)

This example requires some calibration. To make it work, start with an object of a known height and place it in front of the camera. Adjust the pixelsToInches value until the output shows the correct height in inches. After this calibration is done, the Kinect can be used to measure other objects, such as the laptop in Figure 3-6. The measurements will not be perfect because of limitations in the Kinect sensor, but they should provide a decent estimate of height.

Figure 3-6. Example from height calculator program showing that the nearby laptop is 11 inches tall

Note, however, that the objects must be put in the same spot as the original object used for calibration. As an extra credit assignment, the depth values could also be calibrated and used to measure physical distance from the camera. This could then be used to measure an object at a more arbitrary distance from the camera—though this is getting more complicated than is appropriate for this early in the book.

So far, the security camera examples use only one camera. However, it is common to have multiple security cameras, all of which should be monitored at the same time. The following block of example code shows how to do this by combining the output of four cameras into one display window. This code introduces the sideBySide() function, which combines two images together. The options for the sideBySide() function include the name of the image to add to the original image, the side on which place the image (left, right, top, or bottom), and whether or the images should be scaled.

from SimpleCV import Camera, Display
import time

#initialize the IP cameras
cam1 = JpegStreamCamera("http://admin:[email protected]/video.mjpg")  
cam2 = JpegStreamCamera("http://admin:[email protected]/video.mjpg")
cam3 = JpegStreamCamera("http://admin:[email protected]/video.mjpg")
cam4 = JpegStreamCamera("http://admin:[email protected]/video.mjpg")

display = Display((640,480))  

while not display.isDone():
     img1 = cam1.getImage().resize(320, 240)  
     img2 = cam2.getImage().resize(320, 240)
     img3 = cam3.getImage().resize(320, 240)
     img4 = cam4.getImage().resize(320, 240)

     top = img1.sideBySide(img2)  
     bottom = img3.sideBySide(img4)

     combined = top.sideBySide(bottom, side="bottom")  

     combined.save(display)  
     time.sleep(5)

An example of what this might look like is shown in Figure 3-7.

Figure 3-7. A display showing the output from multiple security cameras