Using image sequence

In the previous sections, we have considered how to work with videos stored in video files or captured by cameras. Here we will consider working with the third source of video – image sequences.

Sometimes using ofVideoPlayer for drawing movies is not appropriate. Two main cases are:

  • You need to render a big amount of movies on one screen simultaneously. For example, in a 2D computer arcade game, there can be about 100 moving characters and objects. You can play about 10 different video files in openFrameworks simultaneously, but 100 is too much for it; so some other solution is needed.
  • You need to draw many layers of videos using alpha channel. But at the time of writing this, openFrameworks does not support videos with alpha channel within its standard class ofVideoPlayer. There is support in openFrameworks for only Mac OS X, with class ofQTKitPlayer and example examples/video/osxHighPerformanceVideoPlayerExample. Also, there is an addon ofxAlphaVideoPlayer, but it is not included in the openFrameworks distribution. So some universal solution for video with alpha channel is desirable.

The simplest solution for these cases is using image sequences. Image sequence is a number of images located in the folder and numbered, such as:

image001.png
image002.png
…
image050.png

You can load the images into one array, and draw these one by one, creating a full illusion of movie playing. For resolving the case of large number of videos, you can load many image sequences and draw corresponding images on the screen. For resolving the case of the alpha channel, you just should use images with the alpha channel.

Using image sequences has several constrains:

  • Reading images is a time-consuming operation, so for faster sequence drawing, you should normally read a whole image sequence at testApp::setup().
  • The number of loaded images is limited by the size of the video memory of your graphics card. If you have n images with width w and height h, colored and with alpha channel, the sequence will occupy n * w * h * 4 bytes in the memory. Note that you should use ofTexture for image sequence instead of ofImage, in order to not occupy RAM.

Note

You should never occupy all of your video memory, because some amount of it is needed for usage by your operating system. If you do so during your application execution, your computer can suddenly go slow and even hang on, until you restart it.

Using image sequence example

Till now, there is no standard openFrameworks class for working with image sequences. So we show here a simple example of working with one looped image sequence. The following code loads images from a folder into an array, and then draws these one by one. The index of the currently drawn image is computed using the current time.

Running this example, you will see a toy elephant movie with flying wool cloud. Actually the cloud is an image sequence, drawn over a normal video:

Using image sequence example

The example is based on an emptyExample project in openFrameworks. Before running it, copy the elephant.mov file and the woolCloudSeq folder into the bin/data folder of your project.

Note

This is example 05-Video/06-VideoImageSequence.

In the testApp.h file, inside the testApp class declaration, add declaration of video player object backVideo and image sequence array seq, which is vector of textures:

//Video which will play as the background layer
ofVideoPlayer   backVideo;

//Image sequence which will be overlaid on backVideo
vector<ofTexture> seq;

The setup() function in the testApp.cpp file loads and starts the backVideo movie and reads contents of the data/woolCloudSeq folder into dynamic vector of images seq. The update() function just loads frames of the backVideo movie:

void testApp::setup(){
  //Load background
  backVideo.loadMovie( "elephant.mov" );
  backVideo.play();

  //Set the screen size equal to the backVideo dimensions
  ofSetWindowShape( backVideo.width, backVideo.height );

  //Read image sequence from the folder

  //1. Create object for reading folder contents
  ofDirectory dir;

  //2. Get the number of files in the folder data/woolCloudSeq
  int n = dir.listDir("woolCloudSeq");
  
  //3. Set the array size to n
  seq.resize( n );

  //4. Load images
  for (int i=0; i<n; i++) {
      //Getting i-th file name
      string fileName = dir.getPath( i );

      //Load i-th image
      ofLoadImage( seq[i], fileName );
  }
}

//--------------------------------------------------------------
void testApp::update(){
  backVideo.update();  //Decode the new frame if needed
}

In the preceding code, the list of images in the folder is obtained using ofDirectory dir object. We get the number of files using the following function:

int n = dir.listDir( "woolCloud" );

Calling this function does not only return the amount of files, but also stores the list of file names inside dir. After this, we can read all the file names using the function:

string fileName = dir.getPath( i );

The image sequence is implemented here using vector<ofTexture> seq. Class vector is C++ Standard Template Library container, holding items of any class. It is very good for representing arrays with rare changed sizes. We resize the array with the following function:

seq.resize( n );

Then fill each item, loading the corresponding image from the file:

ofLoadImage( seq[i], fileName );

The visual layering of the image sequence over the backVideo movie is made in the draw() method as follows:

void testApp::draw(){
  //Draw background video
  ofSetColor( 255, 255, 255 );
  backVideo.draw(0,0);

  // Calculate sequence frame number i,
  //based on the current time

  //1. Get the number of seconds from application start
  float time = ofGetElapsedTimef();

  //2. Get the size of image sequence
  int n = seq.size();

  //3. Calculate the sequence duration
  //Our sequence will render 12 frames per second, so:
  float duration = n / 12.0;

  //4. Calculate looped playing position in sequence,
  //in range [0..duration]
  float pos = fmodf( time, duration );

  //5. Convert pos in the frame number
  int i = int( pos / duration * n );

  //Wool cloud will move, so calculate its position
  //depending on time
  float x = ofNoise( time * 0.5 + 10.0 ) * ofGetWidth();
  float y = ofNoise( time * 0.3 + 20.0 ) * ofGetHeight() / 4.0;

  //Enable alpha blending
  ofEnableAlphaBlending();

  //Draw a sequence frame centered at (x,y)
  seq[i].setAnchorPercent( 0.5, 0.5 );
  seq[i].draw( x, y );
}

The most important thing here is computing the image sequence frame number depending on the time. For calculating it, we use the fmod( a, b ) function.

Note

The fmod() function is a float analog of a % operation in C++. For integer a and b, operation a % b returns residue of a divided by b. fmodf( a, b ) does the same, but for float a and b, and the result is float too.

For example, calling fmodf( 3.2, 1.0 ) will return 0.2.

In our case, fmodf( time, duration ) returns the value in range [0, duration), and after reaching the duration value, its result jumps back to 0, giving the necessary looping effect.

Another thing to mention is how we make a cloud move across the screen. To achieve this, we are changing cloud's drawing center coordinates (x, y) for drawing depending on the time. That is, x and y are computed using values of the smoothly changed ofNoise() function, which is a Perlin noise function. See Appendix B, Perlin Noise, for details.

We use different shift values (10.0 and 20.0) in the function in order x and y to change independently. Values 0.5 and 0.3 set the speed of changing x and y. The ofNoise() function returns value in range [0, 1], so finally we multiply the ofNoise results on ofGetWidth() and ofGetHeight() / 4.0 so that the cloud flies in the high quarter of the screen:

float x = ofNoise( time * 0.5 + 10.0 ) * ofGetWidth();
float y = ofNoise( time * 0.3 + 20.0 ) * ofGetHeight() / 4.0;
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset