Processing Media
In the context of the JMF, processing is a broad and encompassing term that includes all manipulation of time-based media. Examples of processing include compressing and decompressing; transcoding, changing between compression formats and adding digital effects; demultiplexing, splitting the media into tracks; multiplexing, combining tracks into a single stream; and rendering, playing back media. It shouldn't be surprising that these examples match the five plug-in categories discussed in the previous section: PlugIns are processing units within the processing chain.
Processing lies at the heart of all programs written to handle media. Although sourcing media with DataSource and MediaLocator objects is a necessity, and perhaps DataSink objects will also be used, processing is the reason for or purpose of the programming. Even playing media, a very common form of JMF program, falls under the umbrella of processing because to play media is to render it.
Not surprisingly then, understanding the JMF model of processing is a necessity for anyone who intends to carry out any significant JMF based programming. At the core of processing sits the Processor class, which will be discussed next. However, the Processor class extends Player, which extends Controller, which extends Clock—all topics covered earlier in the chapter. Similarly, controlling tasks such as transcoding or multiplexing requires knowledge of the JMF classes employed to represent format (both content type and track). Hence to understand processing fully requires understanding not only this section but also the earlier half of the chapter in which these various topics and classes were covered, primarily as preparation to understand processing.
Processor Timescale
Extending Controller as it does indirectly by extending Player, which extends Controller, the Processor class has a similar model of time. Time is stopped or started. But in recognition of the fact that transitioning from stopped to started isn't instantaneous, requiring significant resource acquisition (as it often does), stopped time is subdivided into a number of states. These states reflect the preparedness (or lack thereof) of the Controller to start.
The Controller interface divides stopped time into five states which, in order of least prepared (to start) to prepared are known as Unrealized, Realizing, Realized, Prefetching, and Prefetched.
Because of the nature of processing and, in particular, the need to determine the format of the tracks that compose the media, as well as to specify the processing that will occur on those tracks, the Processor class subdivides stopped time into seven states. The two additional states, known as Configuring and Configured, are added between the Unrealized and Realized states. That leads to a sequence as follows: Unrealized, Configuring, Configured, Realizing, Realized, Prefetching, Prefetched.
A brief summary of each of those states is as follows:
Unrealized— A Processor that has been created but hasn't undertaken any resource gathering.
Configuring— A transition state reflecting the fact that the input (to the Processor) DataSource is being analyzed as to its format and that of its individual tracks.
Configured— A steady-state reflecting a Processor that has successfully gathered format information about the input DataSource. It is in this state that a Processor should be programmed with its processing task.
Realizing— A transition state reflecting the fact that the Processor is gathering information about the resources needed for its task, as well as gathering resources themselves.
Realized— A steady-state reflecting a Processor that has gathered all the non-exclusive resources needed for a task.
Prefetching— A transition state reflecting the fact that the Processor is gathering all resources needed for its task that weren't obtained in the realizing state. Typically, this means acquiring exclusive usage resources such as hardware.
Prefetched— The Processor has acquired all necessary resources, performed all pre-startup processing, and is ready to be started.
Figure 8.29 shows the seven states and their relationship in terms of preparedness to start. The next section details the methods for transitioning between the states as well as the events that a Processor generates as it makes those transitions.
Figure 8.29. The seven states of a Processor as they measure how prepared a Processor is to start.
The addition of the Configuring and Configured states reflects querying the DataSource about its format and that of its constituent tracks. This not only provides finer granularity of stopped time, but also provides the particular instant in which the processor can be programmed. A Processor must be programmed (configured) for what processing it will undertake before it transitions to Realizing because that reflects commitment to a particular type of processing. A Processor cannot be programmed to the task it will undertake until all relevant information about the source media has been obtained. Hence, the Configured state is the only time at which a Processor can be programmed.
Processor Interface
A Processor is a MediaHandler object that extends the Player interface. In many ways, Processor follows a similar abstracted-from-media-details approach as that of Player. Given a particular set of requirements (in this case, processing media as opposed to simply playing it), a suitable Processor object can be obtained by way of the Manager class. From the user-code perspective, this object looks the same as all other Processor objects, possessing the same set of methods. This uniformity of interface, regardless of media particulars, leads to versatile programs capable of processing multiple types of media.
However, media particulars are also where the Processor differs significantly, in a creation sense, from Player. What is entailed in playing media is well understood and requires little specification beyond the media to be played. However, processing is far broader, indeed all but infinite, in variability (for instance, consider different digital effects). The potential with the type of processing that can be performed requires a far tighter prescription from the user at the time of Processor configuration so that the desired form of processing can be achieved. In a nutshell, creating and configuring a Processor is far more complicated than the equivalent task for a Player. The means of Processor creation and configuration, both important and involved, are detailed in a separate subsection.
Figure 8.30 shows the six methods of Processor that are above and beyond those inherited from Player. Three of the methods are vital in the programming of a Processor—configure(), getTrackControls(), and setContentDescriptor(). Two are general information or accessor methods—getContentDescriptor() and getSupportedContentDescriptors(). One, getDataOutput(), is the means of obtaining the output or result of the processing that was performed.
Figure 8.30. The Processor interface.
The configure() method asynchronously brings a Processor to the Configured state. As discussed in the next subsection, this is a vital step in programming a Processor. Only when a Processor is in the Configured state can it be programmed for the processing it will carry out.
The setContentDescriptor() method sets the ContentDescriptor of the DataSource that the Processor outputs. A content type such as AVI, QUICKTIME, or WAVE of the processor's output is set using this method. A content type is also known as meta-format or media container. The method returns the actual ContentDescriptor that was set, which might be the closest matching ContentDescriptor supported by the Processor. The method might also return null if no ContentDescriptor could be set, as well as throwing a NotConfiguredError exception if the Processor is either Unrealized or Configuring.
The getTrackControls() method returns an array of TrackControl objects—one for each track in the DataSource that is being processed. As described in the next subsection, the TrackControl objects can then be employed to program the processing that is performed on the associated track of media. The method throws a NotConfiguredError exception if the Processor isn't at least Configured at the time of calling.
Processors produce a DataSource as the result of the processing they perform. That DataSource can be used as the input to a subsequent Processor or Player. The getDataOutput() method is the means of obtaining the DataSource that a Processor produces. The method throws a NotRealizedError exception if the Processor isn't at least Realized at the time of invocation.
The getContentDescriptor() method returns the currently set ContentDescriptor that will be used for the output of the processor. The getSupportedContentDescriptors() returns an array of all ContentDescriptors that the Processor can output. Both methods throw a NotConfiguredError exception if the Processor isn't at least Configured at the time of calling.
A number of the methods of Processor, most inherited from Controller or Player, exert direct control over the state of a Processor. Others can alter the state as a side effect. The following list shows those methods:
configure()— Moves the Processor from Unrealized to Realized. This is a key step in Processor management because the Configured stage is the only time that a Processor can be programmed as to its task.
realize()— Moves the Processor from either Unrealized or Configured to Realized. Asynchronous (returns immediately). Generally, realize() should only be called after the Processor has reached Configured and has been programmed as to its task.
prefetch()— Moves the Processor from Realized to Prefetched. Asynchronous (returns immediately).
start()— Moves the Processor from its current state to Started. The Processor will transition through any intermediary states and send notice of such transitions to any listeners. As for the realize() method, start() should only be called after a processor has been programmed. Asynchronous (returns immediately).
stop()— Moves the Processor from the Started state back to either Prefetched or Realized, depending on the type of media being processed and the processing task. Synchronous (returns only after Processor is actually stopped).
deallocate()— Deallocates the resources acquired by the Processor and moves it to the Realized state (if at least Realized at the time of call) or to Unrealized (if in Realizing or Unrealized state at time of call). Synchronous.
close()— Closes down the Processor totally. It will post a ControllerClosedEvent. The behavior of any further calls on the Processor is unspecified: the Processor shouldn't be used after it is closed. Synchronous.
setStopTime()— Sets the media time at which it will stop. When the media reaches that time, the Processor will act as though stop() was called. Asynchronous (returns immediately and stops play when time reaches that specified).
setMediaTime()— Changes the media's current time. This might result in a state transition. For instance, a started Processor might return to Prefetching while it refills buffers for the new location in the stream. Asynchronous (in that any state changes triggered aren't waited for).
setRate()— Sets the rate of processing. It might result in a state change on a started Processor for similar reasons to those for setMediaTime(). As for the Player interface, there is no guarantee that a Processor will support any rate other than 1.0. If a requested rate isn't supported, the closest matching rate is set. Asynchronous (in that any state changes triggered aren't waited for).
Figure 8.31 shows the states of a Processor with the methods that force transitions between those states. Figure 8.32 shows the events that a Processor generates as it transitions between states.
Figure 8.31. Methods of Processor that trigger state changes.
Figure 8.32. Events generated when a Processor transitions between states.
Creating and Programming Processors
The key initialization step for a Processor isn't simply its creation but its programming. The specific task that it must perform needs to be defined prior to Processor realization because realization, and the prefetching that follows, involves obtaining the resources necessary to perform the task. However, programming a Processor requires the prior knowledge of the format of the media that will be processed. This apparent dilemma is resolved through the extended model of stopped time provided by Processor: The Configured state provides the opportunity at which the format of the input media is known but prior to the realizing (resource gathering) step.
The JMF provides two primary approaches by which the task of a Processor can be programmed. One, perhaps simpler, approach is to encapsulate all necessary processing as a ProcessorModel object and have the Manager class create a Processor object that will perform exactly that task. The second, arguably more complex but certainly more versatile, approach is to have the Manager class create a Processor but leave programming entirely under user control. The user code must bring the Processor to the Configured state and then use methods and objects such as TrackControls to define the task of the Processor. With that completed, the Processor can be moved through to Started.
The Manager class provides four methods for creating a Processor. Three createProcessor() methods accept the specification of the input media (as a DataSource, URL, or MediaLocator) and return an unrealized Processor. It is the user's responsibility to bring that Processor to the Configured state (with the configure() method), carry out the programming, and then start the Processor. The three createProcessor() methods can throw IOException and NoProcessorException exceptions.
The alternative means of creating a Processor is with Manager's createRealizedProcessor() method. The method accepts a ProcessorModel object that describes all processing requirements—the input and output formats as well as the individual tracks. The method is synchronous (blocking)—it won't return until the Processor is realized. This can make programming considerably easier. As for the other Processor creation methods of Manager, createRealizedProcessor() can throw both IOException and NoProcessorException. It can also throw a CannotRealizeException.
ProcessorModel
The simplest way to program a Processor is with a ProcessorModel object at the time of the Processor's creation using Manager's createRealizedProcessor() method. The ProcessorModel object encapsulates all the processing that the Processor is to perform. In particular, it specifies the input media, the required output content type, and the format of each track of the output.
The task of using a ProcessorModel to obtain an appropriate Processor is relatively simple:
Construct the objects needed by the ProcessorModel constructor (for example, DataSource and Format objects).
Construct the ProcessorModel.
Use Manager's createRealizedProcessor() method to obtain an appropriate Processor.
Figure 8.33 shows the constructors and methods of ProcessorModel. As you can see, the methods of ProcessorModel are simply information queries about the object's attributes. The chief role of the class is to encapsulate all necessary information about the processing task to be performed. Hence the class constructors are the most important elements of the class.
Figure 8.33. The ProcessorModel class.
There are four constructors, although the no-argument constructor is simply a placeholder. The two most commonly used constructors accept either a MediaLocator or DataSource as the specification of the input to the Processor. That is then followed by an array of Format objects—each element of that array specifying the desired format for the particular track. The final argument is a ContentDescriptor specifying the output content type (media container or meta-format). The fourth constructor accepts no specification of the input media: only output track formats and content type. This constructor is used for capturing media (finding a capture device that will meet the output demands of such a processor).
Listing 8.9 shows the use of a ProcessorModel in the apparent transcoding of a movie. A MediaLocator is created specifying the source media, an output content type of MSVIDEO (AVI) is created, and formats are specified for the two tracks: Cinepak for the video and Linear for the audio. Those objects are used to construct a ProcessorModel, which in turn is employed in the creation of a Processor.
Listing 8.9 Use of a ProcessorModel Object in the Apparent Transcoding of a Movie
// Construct objects needed to make ProcessorModel MediaLocator src = new MediaLocator( "file://d:\jmf\book\media\videoexample\iv50_320x240.mov"); formats = new Format[2]; formats[0] = new VideoFormat(VideoFormat.CINEPAK); formats[1] = new AudioFormat(AudioFormat.LINEAR); container = new FileTypeDescriptor(FileTypeDescriptor.MSVIDEO); // Construct the ProcessorModel ProcessorModel model = new ProcessorModel(src,formats,container); // Create the realized Processor using the ProcessorModel Processor p = Manager.createRealizedProcessor(model); |
The one drawback of the ProcessorModel approach to Processor programming is that fine control is removed from the user's hands. In particular, the user has no say over how the processing is performed in terms of what classes are employed. The user cannot specify which PlugIns to employ, and this is a particular drawback in two areas. First, it means that no Effect processing can be performed through a ProcessorModel because a ProcessorModel is only aware of output formats, not manipulations within a format. Second, it might be desirable to perform processing as a chain of PlugIns more complex than Demultiplexer, Codec, and Multiplexer. This is where the second approach to programming a Processor, through its TrackControl objects, comes into its own.
TrackControl
The second means of programming the processing task for a Processor is directly through the TrackControl interface that corresponds to each track composing the media to be processed. The approach is somewhat more involved than using a ProcessorModel but is far more flexible, allowing the user to specify codec chains and renderers to be applied to individual tracks—something not possible with the ProcessorModel approach.
The basic algorithm for pursuing the TrackControl based approach to programming a Processor is as follows:
Create a Processor through the Manager class
Bring the Processor to the Configured state using configure()
Set the required ContentDescriptor on the Processor which specifies the content
type of the media that will be produced
Obtain the TrackControls of the Processor
For each TrackControl
Set the required output Format
If codecs/effects are to be used for the track
Set the TrackControl's codec chain
If the track is to be rendered
Set the TrackControl's renderer
Start the processor
TrackControl is an interface that extends both the FormatControl and Controls interfaces. Figure 8.34 shows the methods of TrackControl plus those inherited from FormatControl (an interface not discussed to date). In terms of programming a Processor, the most important methods are setEnabled(), setFormat(), setCodecChain(), and setRenderer().
Figure 8.34. The TrackControl interface.
The setEnabled() method is used to determine whether a track will be processed. Passing the setEnabled() method a value of false means that the track won't be processed and won't be output by the Processor.
The setFormat() method is passed a Format object specifying the desired output Format that the Processor will produce for that track. The method returns the Format that was actually set (the closest matching Format if the requested Format wasn't supported), or null if no Format could be set.
The setCodecChain() method is passed an array of Codec PlugIns that are to be applied to the track as a chain—that is, in the order they are found in the array. This is a powerful mechanism to exactly controlling the order of compression/decompression and effect processing upon each track. It is important to note that the Effect PlugIn is a subclass of Codec, so these can also be passed as elements of the array. The method might throw an UnSupportedPlugInException or a NotConfiguredError exception.
The setRenderer() method is used to specify the Renderer PlugIn that is to play a particular track. By default, Processors don't render the tracks that they are processing but rather only produce an output DataSource.
It is worth mentioning that the TrackControl interface extends Controls; thus it is possible to obtain the Control objects associated with the track. Among these will be Control objects for any codecs being employed. Thus, it is possible to control the particulars of the compression/decompression on a per-track basis by employing such Control objects.
Listing 8.10 shows the programming of a Processor to perform the same task as the one in Listing 8.9. However, in this case, the Processor is programmed through the TrackControl approach ProcessorModel. As you can see, the setup cost for simple transcoding—what both pieces of code are doing—is higher with the TrackControl approach. What isn't being shown in the fragment is the versatility offered in terms of specifying a codec chain or renderer for each track.
Listing 8.10 Programming of a Processor Through the TrackControls Approach
MediaLocator src = new MediaLocator(
"file://d:\jmf\book\media\videoexample\iv50_320x240.mov");
Processor p = Manager.createProcessor(src);
p.addControllerListener(this);
p.configure();
:
:
public void synchronized controllerUpdate(ControllerEvent e) {
if (e instanceof ConfigureCompleteEvent) {
p.setContentDescriptor(new
FileTypeDescriptor(FileTypeDescriptor.MSVIDEO));
TrackControl[] controls = processor.getTrackControls();
for (int i=0;i<controls.length;i++) {
if (controls[i].getFormat() instanceof VideoFormat)
controls[i].setFormat(new VideoFormat(VideoFormat.CINEPAK));
else
controls[i].setFormat(new AudioFormat(AudioFormat.LINEAR));
}
p.start();
}
else if (e instanceof ...) { ...}
|
Utility Class: MediaStatistics
This subsection discusses a utility class, MediaStatistics, who's source can be found in Listing 8.11 as well as on the book's companion Web site. MediaStatistics employs a Processor in order to acquire information about the format of the individual tracks that compose a media object. A derivative graphical application GUIMediaStatistics, which provides the same functionality but with a graphical user interface, can also be found at the book's companion Web site.
The class itself illustrates a number of the features and classes of JMF that have been discussed in the chapter to date: Processor and Format classes, Controller events, and the asynchronous time model of the JMF.
A MediaStatistics object is constructed by providing the name and location (a String suitable for constructing a MediaLocator) of the media in question. A Processor is created for that media and brought to the Configured state. At that stage, the Format of the individual tracks can be obtained and reported on.
The class also contains a main() method so that it can run as a standalone application. Any command-line options are treated as media names: A MediaStatistics object is constructed and used to generate a report on the format of the tracks of the media. Listing 8.12 shows a run of the application for an MPEG video file. Users should note (as in the example) that the media name must consist of both the protocol (file:// if a local file) and the location of the media (the fully qualified pathname if a local file).
Several features of the class are worth noting. Because discovering the format of tracks requires a Configured Processor, the format information might not be available immediately. To deal with this, the class keeps track of its own state as well as providing methods such as getState() and isKnown() (which returns true when the formats are finally known) so that the user can check when a report is available. Alternatively, and more powerfully, the class could generate its own events and send those to listener classes. Although not difficult to implement, it would obscure the main purpose of the example and hence is omitted.
Two constructors are provided; one returns instantly but makes no guarantees that the format information is available. The second constructor blocks until the format information is known or a prescribed timeout period has transpired. This has the advantage that the user code employing MediaStatistics won't need to wait an unknown period before querying about formats.
The MediaStatistics object keeps track of its own state by catching any exceptions that occur in the Processor creation as well as listening to the Processor. Listening to the Processor in this instance is far simpler than for Players or more sophisticated Processors. Only the ConfigureCompleteEvent is of interest; in which case, the TrackControls should be obtained as a means of obtaining the Format objects for each track. The resources tied up in the Processor can then also be released.
The class contains a number of information (accessor) methods including
getNumTracks()— Gets the number of tracks possessed by the media
getState()— Determines the state of the object
getReport()— Gets a String detailing the format of all tracks
getTrackFormat()— Returns the Format object for the specified track number
isAudioTrack() and isVideoTrack()— Return true if the specified track number is an audio or video track
isKnown()— Returns true if the format information is known (if the object's state is KNOWN).
Listing 8.11 The MediaStatistics Application that Produces Statistics About the Particular Format of Media
import javax.media.*;
import javax.media.control.*;
import javax.media.format.*;
/******************************************************************
* A Class to determine statistics about the tracks that compose
* a media object. Given the name (URL/location) of media a
* Processor is constructed and brought to the Configured state.
* At that stage its TrackControls are obtained as a means of
* discovering the Formats of the individual tracks.
*
* Because reaching Configured can take time, the MediaStatistics
* object keeps track of its own state and provides methods for
* determining that state. Only when it reaches the KNOWN state
* can statistics be obtained. Similarly there are 2 constructors:
* one creating a Processor and starting it toward Configured but
* returning immediately. The other is a blocking constructor, it
* won't return until the Processor reaches Configured or the
* specified time-out expires. This has the advantage that the
* object can be used immediately (rather than polling it to
* determine when it enters the KNOWN state).
*
* The chief information gathering method is getReport() which
* returns a String reporting on the Format of all tracks of
* the media. Alternatively the Format of individual tracks can
* also be ascertained.
*
* @author Spike Barlow
******************************************************************/
public class MediaStatistics implements ControllerListener {
/** State: Yet to create the Processor. */
public static final int NOT_CREATED = 0;
/** State: Unable to create the Processor. */
public static final int FAILED = -1;
/** State: Processor is Configuring. */
public static final int CONFIGURING = 1;
/** State: Details of media are Known. */
public static final int KNOWN = 2;
/** Number of tracks is Unknown. */
public static final int UNKNOWN = Integer.MIN_VALUE;
/** Period in milliseconds to sleep for before
* rechecking if reached KNOWN state. */
protected static final int WAIT_INTERVAL = 20;
/** Number of tracks possessed by the media. */
protected int numTracks = UNKNOWN;
/** Formats of the individual tracks. */
protected Format[] trackFormats;
/** Processor needed to ascertain track information. */
protected Processor processor;
/** State that the object is currently in. A reflection
* of the state the Processor is in. */
protected int state = NOT_CREATED;
/** The name of the media on which stats are being compiled. */
protected String nameOfMedia;
/**************************************************************
* Construct a MediaStatistics object for the media with the
* passed name. This is a blocking constructor. It returns
* only when it is possible to obtain the track statistics or
* when the specified time-out period (in milliseconds) has
* transpired.
***************************************************************/
MediaStatistics(String mediaName, int timeOutInMilliseconds) {
nameOfMedia = mediaName;
// Construct the Processor
try {
MediaLocator locator = new MediaLocator(mediaName);
processor = Manager.createProcessor(locator);
}
// Any exception is a failure.
catch (Exception e) {
state = FAILED;
return;
}
// Listen to and start configuration of the Processor.
processor.addControllerListener(this);
state = CONFIGURING;
processor.configure();
//////////////////////////////////////////////////////////
// Wait till the Processor reaches configured (the object
// reaches KNOWN) or the specified time-out interval has
// transpired, by looping, sleeping,and rechecking.
//////////////////////////////////////////////////////////
if (timeOutInMilliseconds>0) {
int waitTime = 0;
while (waitTime<timeOutInMilliseconds && !isKnown()) {
try { Thread.sleep(WAIT_INTERVAL); }
catch (InterruptedException ie) { }
waitTime += WAIT_INTERVAL;
}
}
}
/**************************************************************
* Construct a MediaStatistics object for the media with the
* passed name. This is not a blocking constructor: it returns
* immediately. Thus calling getReport() immediately may result
* in "Still parsing media" report. The isKnown() method should
* be used to check for this condition.
****************************************************************/
MediaStatistics(String mediaName) {
this(mediaName,-1);
}
/**************************************************************
* Respond to events from the Porcessor. In particular the
* ConfigureComplete event is the only one of interest. In this
* case obtain the TrackControls anduse these to obtain the
* Formats of each track. Also modify the state and close down
* the Processor (free up its resources).
***************************************************************/
public synchronized void controllerUpdate(ControllerEvent e) {
if (e instanceof ConfigureCompleteEvent) {
TrackControl[] controls = processor.getTrackControls();
// As long as there are TrackControls, get each track's format.
if (controls.length!=0) {
numTracks = controls.length;
trackFormats = new Format[controls.length];
for (int i=0;i<controls.length;i++) {
trackFormats[i] = controls[i].getFormat();
}
state = KNOWN;
}
else {
state = FAILED;
}
// Close down the Processor.
processor.removeControllerListener(this);
processor.close();
processor = null;
}
}
/***********************************************************
* Determine what state the object is in. Returns one of the
* class constants such as KNOWN, FAILED or CONFIGURING.
************************************************************/
public int getState() {
return state;
}
/************************************************************
* Determine the number of tracks possessed by the media. If
* that is unknown, either due to the processor creation
* failing or because the processor is not yet Configured then
* the class constant UNKNOWN is returned.
*************************************************************/
public int getNumTracks() {
return numTracks;
}
/*************************************************************
* Obtain the Format for the specified track number. If the
* track doesn't exist, or it has yet to be determined how
* many tracks the media possesses, null is returned.
**************************************************************/
public Format getTrackFormat(int track) {
if (track<0 || track>=numTracks)
return null;
return trackFormats[track];
}
/***************************************************************
* Is the object in the KNOWN state? The KNOWN state reflects
* the fact that information is known about the number and
* Format of the tracks. The method can be used to ascertain
* whether a report is available (meaningful) or not.
****************************************************************/
public boolean isKnown() {
return state==KNOWN;
}
/***************************************************************
* Returns true if the specified track number is an audio track.
* If the track doesn't exist, the number of tracks is yet
* unknown, or it isn't audio then false is returned.
****************************************************************/
public boolean isAudioTrack(int track) {
if (track<0 || track>=numTracks)
return false;
return trackFormats[track] instanceof AudioFormat;
}
/***************************************************************
* Returns true if the specified track number is a video track.
* If the track doesn't exist, the number of tracks is yet
* unknown, or it isn't video then false is returned.
****************************************************************/
public boolean isVideoTrack(int track) {
if (track<0 || track>=numTracks)
return false;
return trackFormats[track] instanceof VideoFormat;
}
/*****************************************************************
* Returns a report, as a String, detailing thenumber and format
* of the individual tracks that compose the media that this
* object obtained statistics for. If the object is not in the
* KNOWN state then the report is a simple String, indicating
* this.
******************************************************************/
public String getReport() {
String mess;
if (state==FAILED)
return "Unable to Handle Media " + nameOfMedia;
else if (state==CONFIGURING)
return "Still Parsing Media " + nameOfMedia;
else if (state==KNOWN) {
if (numTracks==1)
mess = nameOfMedia + ": 1 Track
";
else
mess = nameOfMedia + ": " + numTracks + " Tracks
";
for (int i=0;i<numTracks;i++) {
if (trackFormats[i] instanceof AudioFormat)
mess += " "+(i+1)+" [Audio]: ";
else if (trackFormats[i] instanceof VideoFormat)
mess += " "+(i+1)+" [Video]: ";
else
mess += " "+(i+1)+" [Unknown]: ";
mess += trackFormats[i].toString() + "
";
}
return mess;
}
else
return "Unknown State in Processing " + nameOfMedia;
}
/*************************************************************
* Simple main method to exercise the class. Takes command
* line arguments and constructs MediaStatistics objects for
* them, before generating a report on them.
**************************************************************/
public static void main(String[] args) {
MediaStatistics[] stats = new MediaStatistics[args.length];
for (int i=0;i<args.length;i++) {
stats[i] = new MediaStatistics(args[i],200);
System.out.println(stats[i].getReport());
stats[i] = null;
}
System.exit(0);
}
}
|
Listing 8.12 Output of a Run of the MediaStatistics Program on a Particular MPEG Movie File
C:SpikeJMFCode>java MediaStatistics file://d:mediaadday.mpeg
file://d:mediaadday.mpeg: 2 Tracks
1 [Video]: MPEG, 176x120, FrameRate=29.9, Length=31680
2 [Audio]: mpegaudio, 44100.0 Hz, 16-bit, Mono, LittleEndian,
Signed, 4000.0 frame rate, FrameSize=16384 bits
|
Utility Class: Location2Location
This subsection discusses a utility class, Location2Location, found in Listing 8.13 as well as on the book's companion Web site. Location2Location is a class that both transcodes media to a different format and saves/writes the resulting media to a new location.
In order to perform its task, Location2Location combines many of the most important classes and features of the JMF, and is the most comprehensive example of the JMF features to date in the book. Key classes involved include Processor, Manager, DataSource, DataSink, ProcessorModel, and MediaLocator; whereas the event-driven asynchronous nature of the JMF is illustrated through the class implementing ControllerListener while also using an anonymous class to listen for events from the DataSink object.
The class is employed through the mechanism of the user constructing a Location2Location object that specifies the origin and destination of the media (either Strings or MediaLocators), the desired format for the individual tracks that compose the media (an array of Formats), and the content type (ContentDescriptor) for the resulting media container. Construction automatically initiates the transcoding portion of the task, whereas transfer to the new destination is achieved by calling the transfer() method.
The transcoding tasks— transforming the format of the media—is achieved via the processor class, whereas the writing to a new location is achieved through a DataSink. The Processor object is created through the specification of a ProcessorModel that includes the Format specification for the individual tracks as well as the content type (ContentDescriptor) of the resulting container (for example, Quicktime or AVI).
The DataSink is constructed using the DataSource that is the output of the Processor together with the user specified destination.
Both the Processor and DataSink perform their respective tasks (transcoding and rendering to a destination) asynchronously. The class illustrates means of dealing with this pervasive feature of the JMF. The class listens for events from both the Processor and DataSink and maintains its own internal model of its current state. This is exposed to the user through a number of class constants and a method getState(), which is used to query the current state. Further, the transfer() method can be invoked asynchronously, which will return immediately, but the user will need to check periodically for when the process is complete. It can also be invoked synchronously, which will block until the process completes. An even better solution would be for Location2Location to generate its own events and allow classes to register themselves as listeners for those events. Such an approach isn't difficult to implement, but requires the writing of an additional interface (the Listener interface) and class (the Event class) as well as the code to maintain the list of listeners. We feel that this further detail would obscure the purpose of the example, which is to illustrate features of the JMF.
Listing 8.13 The Location2Location Utility Class Capable of Transferring and Transcoding Media
import javax.media.*;
import javax.media.datasink.*;
import javax.media.protocol.*;
/***************************************************************
* Transfer media from one location to another carrying out the
* specified transcoding (track formats and content type) at the
* same time.
*<p>Users specify a source and destination location, the
* Formats (to be realised) of the individual tracks, and a
* ContentDescriptor (content type) for output.
*<p>A Processor is created to perform and transcoding and its
* output DataSource is employed to construct a DataSink in
* order to complete the transfer.
*<p>The most important method of the class is transfer() as
* this opens and starts the DataSink. The constructor builds
* both the Processor (which is starts) and the DataSink.
*<p>The object keeps track of its own state, which can be queried
* with the getState() method. Defined constants are FAILED,
* TRANSLATING, TRANSFERRING, and FINISHED. The process is
* asychronous: transcoding largish movies can take a long time.
* The calling code should make allowances for that.
****************************************************************/
public class Location2Location implements ControllerListener {
/** Output of the Processor: the transcoded media. */
protected DataSource source;
/** Sink used to "write" out the transcoded media. */
protected DataSink sink;
/** Processor used to transcode the media. */
protected Processor processor;
/** Model used in constructing the processor, and which
* specifies track formats and output content type */
protected ProcessorModel model;
/** State the object is in. */
protected int state;
/** Location that the media will be "written" to. */
protected MediaLocator sinkLocation;
/** The rate of translation. */
protected float translationRate;
/** Process has failed. */
public static final int FAILED = 0;
/** Processor is working but not finished. DataSink is yet
* to start. */
public static final int TRANSLATING = 1;
/** DataSink has started but not finished. */
public static final int TRANSFERRING = 3;
/** Transcoding and transfer is complete. */
public static final int FINISHED = 4;
/** String names for each of the states. More user friendly */
private static final String[] STATE_NAMES = {
"Failed", "Translating", "<UNUSED>", "Transferring",
"Finished"};
/** Period (in milliseconds) between checks for the blocking
* transfer method. */
public static final int WAIT_PERIOD = 50;
/** Wait an "indefinite" period of time for the transfer
* method to complete. i.e., pass to transfer() if the
* user wishes to block till the process is complete,
* regardless of how long it will take. */
public static final int INDEFINITE = Integer.MAX_VALUE;
/*******************************************************************
* Construct a transfer/transcode object that transfers media from
* sourceLocation to destinationLocation, transcoding the tracks as
* specified by the outputFormats. The output media is to have a
* content type of outputContainer and the process should (if
* possible) run at the passed rate.
*********************************************************************/
Location2Location(MediaLocator sourceLocation,
MediaLocator destinationLocation, Format[] outputFormats,
ContentDescriptor outputContainer, double rate) {
//////////////////////////////////////////////
// Construct the processor for the transcoding
//////////////////////////////////////////////
state = TRANSLATING;
sinkLocation = destinationLocation;
try {
if (sourceLocation==null)
model = new ProcessorModel(outputFormats,outputContainer);
else
model = new ProcessorModel(sourceLocation,
outputFormats,outputContainer);
processor = Manager.createRealizedProcessor(model);
}
catch (Exception e) {
state = FAILED;
return;
}
translationRate = processor.setRate((float)Math.abs(rate));
processor.addControllerListener(this);
////////////////////////////////////////////////////////////
// Construct the DataSink and employ an anonymous class as
// a DataSink listener in order that the end of transfer
// (completion of task) can be detected.
///////////////////////////////////////////////////////////
source = processor.getDataOutput();
try {
sink = Manager.createDataSink(source,sinkLocation);
}
catch (Exception sinkException) {
state = FAILED;
processor.removeControllerListener(this);
processor.close();
processor = null;
return;
}
sink.addDataSinkListener(new DataSinkListener() {
public void dataSinkUpdate(DataSinkEvent e) {
if (e instanceof EndOfStreamEvent) {
sink.close();
source.disconnect();
if (state!=FAILED)
state = FINISHED;
}
else if (e instanceof DataSinkErrorEvent) {
if (sink!=null)
sink.close();
if (source!=null)
source.disconnect();
state = FAILED;
}
}
});
// Start the transcoding
processor.start();
}
/***************************************************************
* Alternate constructor: source and destination specified as
* Strings, and no rate provided (hence rate of 1.0)
****************************************************************/
Location2Location(String sourceName, String destinationName,
Format[] outputFormats, ContentDescriptor outputContainer) {
this(new MediaLocator(sourceName), new MediaLocator(destinationName),
outputFormats, outputContainer);
}
/****************************************************************
* Alternate constructor: No rate specified therefore rate of 1.0
*****************************************************************/
Location2Location(MediaLocator sourceLocation,
MediaLocator destinationLocation, Format[] outputFormats,
ContentDescriptor outputContainer) {
this(sourceLocation,destinationLocation,outputFormats,outputContainer,1.0f);
}
/***************************************************************
* Alternate constructor: source and destination specified as
* Strings.
****************************************************************/
Location2Location(String sourceName, String destinationName,
Format[] outputFormats, ContentDescriptor outputContainer,
double rate) {
this(new MediaLocator(sourceName), new MediaLocator(destinationName),
outputFormats, outputContainer, rate);
}
/**************************************************************
* Respond to events from the Processor performing the transcoding.
* If its task is completed (end of media) close it down. If there
* is an error close it down and mark the process as FAILED.
*****************************************************************/
public synchronized void controllerUpdate(ControllerEvent e) {
if (state==FAILED)
return;
// Transcoding complete.
if (e instanceof StopEvent) {
processor.removeControllerListener(this);
processor.close();
if (state==TRANSLATING)
state = TRANSFERRING;
}
// Transcoding failed.
else if (e instanceof ControllerErrorEvent) {
processor.removeControllerListener(this);
processor.close();
state = FAILED;
}
}
/***********************************************************
* Initiate the transfer through a DataSink to the destination
* and wait (block) until the process is complete (or failed)
* or the supplied number of milliseconds timeout has passed.
* The method returns the total amount of time it blocked.
*************************************************************/
public int transfer(int timeOut) {
// Can't initiate: Processor already failed to transcode
////////////////////////////////////////////////////////
if (state==FAILED)
return -1;
// Start the DataSink
//////////////////////
try {
sink.open();
sink.start();
}
catch (Exception e) {
state = FAILED;
return -1;
}
if (state==TRANSLATING)
state = TRANSFERRING;
if (timeOut<=0)
return timeOut;
// Wait till the process is complete, failed, or the
// prescribed time has passed.
/////////////////////////////////////////////////////
int waited = 0;
while (state!=FAILED && state!=FINISHED && waited<timeOut) {
try { Thread.sleep(WAIT_PERIOD); }
catch (InterruptedException ie) { }
waited += WAIT_PERIOD;
}
return waited;
}
/***************************************************
* Initiate the transfer through a DataSink to the
* destination but return immediately to the caller.
****************************************************/
public void transfer() {
transfer(-1);
}
/***************************************************
* Determine the object's current state. Returns one
* of the class constants.
****************************************************/
public int getState() {
return state;
}
/***************************************************
* Returns the object's state as a String. A more
* user friendly version of getState().
****************************************************/
public String getStateName() {
return STATE_NAMES[state];
}
/***************************************************
* Obtain the rate being used for the process. This
* is often 1, despite what the user may have supplied
* as Clocks (hence Processors) don't have to support
* any other rate than 1 (and will default to that).
****************************************************/
public float getRate() {
return translationRate;
}
/***************************************************
* Set the time at which media processing will stop.
* Specification is in media time. This means only
* the first "when" amount of the media will be
* transferred.
****************************************************/
public void setStopTime(Time when) {
if (processor!=null)
processor.setStopTime(when);
}
/***************************************************
* Stop the processing and hence transfer. This
* gives user control over the duration of a
* transfer. It could be started with the transfer()
* call and after a specified period stop() could
* be called.
****************************************************/
public void stop() {
if (processor!=null)
processor.stop();
}
}
|
As a means of illustrating how the Location2Location class might be employed, the simple example StaticTranscode found in Listing 8.14 is provided. The class provides no user interaction, performing the same task each time. That task is to transcode two media files into two others. The first media is a short piece of audio (someone playing an electric guitar) that is transcoded from linear, wave format into MP3. The second media is a movie consisting of both audio (GSM format) and video (Indeo 5.0 format) in a Quicktime meta-format, which is transcoded into AVI meta-format with linear audio and Cinepak video. Users wanting to perform transcoding could use StaticTranscode as a starting template, modifying filenames and formats appropriately. Alternatively, for one-off tasks, JMStudio could be used interactively.
As you can see by viewing the source, employing the Location2Location class requires the prior construction of Format and ContentDescriptor objects that detail the format for the transcoding that will occur.
StaticTranscode is a trivial example, simply illustrating how Location2Location might be used. However it is relatively easy to write applications that build on top of the functionality provided by Location2Location. For example, an audio ripper (or its inverse)—a program that converts CD audio to MP3—could easily be written so that given a directory, it processes all files found there and converts them into MP3.
Listing 8.14 The StaticTranscode Class, a Simple Example of How the Location2Location Class Might Be Employed
import javax.media.*;
import javax.media.protocol.*;
import javax.media.format.*;
/*****************************************************************
* Simple example to show the Location2Location class in action.
* The Location2Location class transfer media from one location to
* another performing any requested tanscoding (format changes)
* at the same time.
*
* The class is used twice. Once to transform a short wave audio
* file of an electric guitar (guitar.wav) into MP3 format.
* The second example converts of Quicktime version of the example
* video from chapter 7, encoded with the Indeo 5.o codec and
* GSM audio into an AVI version with Cinepak codec for the video
* and linear encoding for the audio.
******************************************************************/
public class StaticTranscode {
public static void main(String[] args) {
String src;
String dest;
Format[] formats;
ContentDescriptor container;
int waited;
Location2Location dupe;
/////////////////////////////////////////////////////////////////
// Transcode a wave audio file into an MP3 file, transferring it
// to a new location (dest) at the same time.
////////////////////////////////////////////////////////////////
src = "file://d:\jmf\book\media\guitar.wav";
dest = "file://d:\jmf\book\media\guitar.mp3";
formats = new Format[1];
formats[0] = new AudioFormat(AudioFormat.MPEGLAYER3);
container = new FileTypeDescriptor(FileTypeDescriptor.MPEG_AUDIO);
dupe = new Location2Location(src,dest,formats,container);
System.out.println("After creation, state = " + dupe.getStateName());
waited = dupe.transfer(10000);
System.out.println("Waited " + waited + " milliseconds. State is"
+ " now " +dupe.getStateName());
///////////////////////////////////////////////////////////////////
// Transcode a Quicktime version of a movie into an AVI version.
// The video codec is altered from Indeo5.0 to Cinepak,the audio
// track is transcoded from GSM to linear, and is result is saved
// as a file "qaz.avi".
//////////////////////////////////////////////////////////////////
src = "file://d:\jmf\book\media\videoexample\iv50_320x240.mov";
dest = "file://d:\jmf\book\media\qaz.avi";
formats = new Format[2];
formats[0] = new VideoFormat(VideoFormat.CINEPAK);
formats[1] = new AudioFormat(AudioFormat.LINEAR);
container = new FileTypeDescriptor(FileTypeDescriptor.MSVIDEO);
dupe = new Location2Location(src,dest,formats,container,5.0f);
System.out.println("After creation, state = " + dupe.getStateName());
waited = dupe.transfer(Location2Location.INDEFINITE);
int state = dupe.getState();
System.out.println("Waited " + (waited/1000) + " seconds. State is"
+ " now " +dupe.getStateName() + ", rate was " + dupe.getRate());
System.exit(0);
}
}
|
The output of StaticTranscode is shown in Listing 8.15. As you can see, the audio file, 6.5 seconds in length, took just over 2.5 seconds to transcode and save. On the other hand, the movie of less than 1 minute (57 seconds) took more than 10 minutes to transcode and save. This was on a Pentium IV system with 256MB of RAM. Clearly these processes can take a long time to complete. Furthermore, the actual time required varies depending on other factors such as system load at the time. Running the same program again saw variations of as much as 20% in time to complete.
Listing 8.15 Output of the StaticTranscode Program Showing Time Needed to Transcode and Sink the Media
After creation, state = 1 Waited 2660 milliseconds. State is now 4 After creation, state = 1 Waited 618 seconds. State is now 4, rate was 1.0 |