Using video footage is an easy way to add dynamic layers to an interactive project scene. And processing video is the basis of modern computer-generated video art. This chapter will cover the basic and advanced topics on playing, layering, and processing videos in the openFrameworks projects:
- Playing a video file
- Processing video frames
- Radial and horizontal slit-scan effects
- Processing a live video from the camera
- The video synthesizer
- Using image sequences
Video is the most usable container for dynamic media today. It consists of a number of frames—moving images and soundtrack—all encapsulated in a single file. In principle, each video can be represented by a sequence of separate image files for each frame and audio files for soundtracks. But using a single file is often more comfortable.
Tip
The big advantage of using a single video file is that modern video codecs can significantly reduce the video's file size compared to the size of image sequence plus the soundtrack file sizes. Also, decoding video can be made easier using GPU instead of CPU. These two reasons give a possibility to play HD videos smoothly and at high framerates with openFrameworks. Nevertheless, in Using image sequence section, we will see when using an image sequence is more suitable than using a single video file.
The best known file formats for video are AVI, MP4, and MOV. All these formats are supported in openFrameworks, but in general it is preferable to use the MOV format because openFrameworks uses Apple QuickTime SDK for playing video in Mac OS X and Windows, and the MOV format is native for QuickTime.
Note
To play videos in openFrameworks in Windows, you need to install QuickTime from http://www.apple.com/quicktime/download/.
If you want to use a video from sources such as youtube.com or vimeo.com, you can download it using services such as keepvid.com. Be careful about considering the licensee limitations when using these videos.
If you create a video file using video editors such as Adobe AfterEffects, or maybe in 3D software such as Autodesk 3DSMax, you should be aware of choosing the right codec while saving your video file. Here is the list of possible usages and corresponding codecs:
- If you want to play the video as a normal video footage, without speed changing and using alpha channel, the best option for you is using some MPEG4 codec (such as H.264), with large quality value settings in codec. It will have a good quality and a small file size. Such a codec may be crucial in case of using HD videos, because using other codecs can give a huge file, which is hard to read in real time from the disk.
Remember, MPEG4 codecs do "deep" video compression, and use many previous frames for decoding a new frame; so changing the direction and speed of playing such videos can have a negative impact on the performance of your application.
- If you want to use video as a clip for VJ-ing, consider using codec with (Apple) Motion-JPEG.
- If you need a video with an alpha channel, use Apple-PNG format and set Millions+ colors or 32-bit color mode. With this setting, the file size will be larger when using Motion-JPEG, but will give very good quality and good performance. At the time of writing this book, openFrameworks's use is limited when using such videos. Please see details in the Using image sequence section.