The way to input sound data from a microphone or other audio input device is similar to the sound output considered earlier, with small changes:
testApp
class declaration as follows:ofSoundStream soundStream; void audioReceived( float *input, int bufferSize, int nChannels );
testApp::setup()
function definition, add the following:soundStream.setup( this, 0, 1, 44100, 512, 4 );
Here, this
is a pointer to our testApp
object which will receive the microphone's sound data by calling our testApp::audioReceived
function.
Subsequently, 0
is the number of output channels (hence, no output), 1
is the number of input channels (hence, mono input), 44100
is a sample rate, that is, the received number of audio samples per second.
The last two parameters 512
and 4
are the size of the buffer for audio samples and the number of buffers.
void testApp::audioReceived( float *input, int bufferSize, int nChannels ){ //... use input array here }
This is a function that can use the input
array of the audio sample's data. It is a function which actually processes sound.
Values of input
lie in the range from -1.0
to 1.0
. The size of the input
is equal to the bufferSize * nChannels
, and samples in the channels are interleaved. Namely, if nChannels = 2
, then this is a stereo signal, so input[0]
and input[1]
mean the first audio samples for the left and right channels. Correspondingly, input[2]
and input[3]
mean the second audio samples, and so on.
Functions listDevices()
, setDeviceID( id )
, stop()
, start()
, and close()
, discussed in the Generating sounds section are applicable here as well.
Note that the audioReceived()
function is called by openFrameworks independent of calling the update()
and draw()
functions. Namely, it is called when the next buffer with audio samples is received from the sound card:
Let's see an example of using sound input.
Sound samplers are music devices which record and replay sound samples. Let's make a simple sampler which records and plays one short sound. Recording and playing are made in a loop. Additionally, the current sound is drawn as a graph and also as a stripe image, so you can use it for making input pictures for an image-to-sound transcoder example as described earlier.
This example is based on the emptyExample
project in openFrameworks. As it receives and generates sound, add the next code to testApp.h
in the class testApp
declaration:
//Function for receiving audio void audioReceived( float *input, int bufferSize, int nChannels ); //Function for generating audio void audioOut( float *input, int bufferSize, int nChannels ); //Object for sound output and input setup ofSoundStream soundStream;
At the beginning of testApp.cpp
, after the #include "testApp.h"
line, add constants and variables. The main thing here is the buffer
array, which is used as storage for sound recording and also as a data source for sound playing. This buffer has a size which is generally not equal to the size of the input
and output
buffers passed in the audioReceived()
and audioOut()
functions. This means we need to use variables for holding the current recording and playing position, recPos
and playPos
respectively. Also, there are two modes of work selectable from the keyboard, sample recording and playing.To manage this, we define the flags recordingEnabled
and playingEnabled
:
//Constants const int sampleRate = 44100; //Sample rate of sound const float duration = 0.25; //Duration of the recorded //sound in seconds const int N = duration * sampleRate; //Size of the PCM buffer //Variables vector<float> buffer; //PCM buffer of sound sample int recPos = 0; //Current recording position in the buffer int playPos = 0; //Current playing position in the buffer int recordingEnabled = 1; //Is recording enabled int playingEnabled = 0; //Is playing enabled
The setup()
function sets the buffer
size and starts the sound output and input:
void testApp::setup(){ //Set buffer size and fill it by zeros buffer.resize( N, 0.0 ); //Start the sound output in stereo (2 channels) //and sound input in mono (1 channel) soundStream.setup( this, 2, 1, sampleRate, 256, 4 ); }
In order to record sound updates on the screen more rapidly, we set the sound card buffer size equal to 256
(not 512
as in the previous examples). If you hear some clicks in the sound, it means that the application is too expensive for your computer (due to CPU or sound card), so increase the value back to 512
.
The update()
function is empty. The draw()
function draws a graph of the buffer and its stripe image. A stripe image is drawn by vertical lines of different colors. Note that the width of the screen w
= 1024
is less than the sound buffer size N
, so we are shrinking the buffer on the screen using the conversion formula:
i = float(x) * N / w
Also, while converting the buffer values into color values of the lines, we apply square root transformation. It makes small changes of audio samples more visually distinct. Image-to-sound transcoding works better with such a transformed image too. See the draw()
function code in the example's text.
The most important functions of the example are audioReceived()
and audioOut()
. They record and play audio samples to and from the buffer
array:
//Audio input void testApp::audioReceived( float *input, int bufferSize, int nChannels ) { //If recording is enabled by the user, //then store received data if ( recordingEnabled ) { for (int i=0; i<bufferSize; i++) { buffer[ recPos ] = input[i]; recPos++; //When the end of buffer is reached, recPos sets //to 0, so we record sound in a loop recPos %= N; } } } //Audio output void testApp::audioOut( float *output, int bufferSize, int nChannels) { //If playing is enabled by the user, then do output sound if ( playingEnabled ) { for (int i=0; i<bufferSize; i++) { output[ 2*i ] = output[ 2*i+1 ] = buffer[ playPos ]; playPos++; //When the end of buffer is reached, playPos sets //to 0, so we hear looped sound playPos %= N; } } }
Finally, add a reaction on the keyboard. Keys 1 and 2 will switch between the recording and playing modes. Key S will save the screen image to the grab.png
file, so you can print it and use it as an input in the Image-to-sound transcoder example:
void testApp::keyPressed( int key ){ //Enable recording mode if ( key == '1' ) { recordingEnabled = 1; playingEnabled = 0; } //Enable playing mode if ( key == '2' ) { recordingEnabled = 0; playingEnabled = 1; } //Save screen image to the file if ( key == 's' ) { ofImage grab; grab.grabScreen( 0, 0, ofGetWidth(), ofGetHeight() ); grab.saveImage( "grab.png" ); } }
If you use a laptop, then it is highly possible that you may have an built-in microphone. If not, then before running the application, connect a microphone or web camera to your computer (most web cameras have microphones). Run the application and say something into the microphone. You will see a graph of the sound and also a stripe image as shown in the following screenshot:
Continue talking into the microphone and press 2. The recording will be stopped and the last recorded sound sample will play in a loop. As the recording is performed in a looped buffer with a length duration
= 0.25 seconds, the resulting duration of the sample will be 0.25 seconds too.
Now press S and the image will be saved into the grab.png
file in the bin/data
folder of the application. Press 1 and the recording will start again.
If your application hangs at the start, you might have selected an improper sound device. For example, it can happen if you run the application in Microsoft Windows 7 and no input device is connected. Also, some sound devices are output or input only. Use soundStream.listDevices()
to show the list of available devices, and then call soundStream.setDeviceID( id )
with proper id
. If you use an input-only device, try to run the example with sound output disabled. To do so, change the number of output channels from 2
to 0
in soundStream.setup()
calling in the testApp::setup()
function:
soundStream.setup( this, 0, 1, sampleRate, 256, 4 );
Let's talk about saving the recorded sound sample into a file. Currently, openFrameworks does not have a function for writing sound samples to WAV or MP3 files. To do so, you need to download and use an addon such as ofxSndFile.
Another simple but not so comfortable way is to save your file as a RAW file containing only audio samples, and then open it in an audio editor, specifying its sample rate and audio samples' type. To do so in the preceding example, add the following code to the keyPressed( int key )
function:
//Write the sound sample to raw-file if ( key == 'f' ) { //Create a file for writing //Here "wb" means that we open binary file for writing FILE *file = fopen( ofToDataPath("sound.raw").c_str(), "wb" ); //Write the buffer into file fwrite( &buffer[0], N, sizeof( buffer[0] ), file ); //Close the file fclose( file ); }
Now if you run the application and press F, the current sound sample will be written to the sound.raw
file in the bin/data
folder of the application. You can open it in a sound editor such as Audacity (free) or Sony Sound Forge (commercial) and while opening, specify the sample parameters:
Finally, using the editor, you can save it as a WAV or an MP3 file.
Now we will consider how to get meaningful information from sound and use it for adding a real-time reaction to sound into your projects.