Extending the JMF

Although the JMF provides support for an impressive number of formats (codecs), content types, and protocols, its coverage isn't complete. Although Sun's JMF team has pledged to support new open standard codecs and other similar advances in the area of time-based media, there will continue to be gaps between the coverage of the distributed JMF and the totality of time-based media. Several reasons for this difference are as follows:

Lag— Newer standards (for example, MPEG-4) can take some time to be implemented efficiency, tested fully, and brought into the JMF stable.

Proprietary— JMF is a free, open-standard with several complete implementations. The JMF couldn't be free and truly platform independent if it were encumbered by proprietary formats that have restrictions imposed on their use (for example, a fee per use of a particular codec).

Specialization— The JMF team at Sun has only limited resources, whereas the time-based media area is very large and continually advancing. Implementing more commonly used formats will clearly have higher priority than specialist niche formats, implying that specialist formats can take considerable time to appear.

Fortunately, the JMF is purposely built to be extended by users. Users can write their own codecs, multiplexers, demultiplexers, players, data sources, effects, and so on. These can then be seamlessly incorporated into the infrastructure that the JMF provides. They are then automatically (that is, through the central manager classes) available for subsequent usage.

Hence, for instance, an online community employing a particular form of time-based media might (over time) implement a Codec, Multiplexer, Demultiplexer, Renderer, and DataSource in support of that new format. These could then be seamlessly integrated into the JMF (and indeed distributed to the wider JMF-using community so that users also have automatic support for the new type of media), allowing full playing and processing of the media. Alternatively, a company might sell an AV conferencing system that incorporates custom hardware for compression and decompression but who's software side is implemented in the JMF. By writing custom Player and Processor (or PlugIn) classes, the company can use the JMF infrastructure and paradigm of DataSources, Players, and Processors. However those custom classes would ensure that the system employs the hardware component—reaping the benefits of speed offered by the hardware. Yet another example is that a library of digital effects for both audio and video might be maintained on some community site. These effects would be implementations of the Effect PlugIn. Users wanting to employ an effect could simply download it from the repository and add it to their JMF installation on their machine. Those wanting to add a new effect (for example, motion blur) could implement a suitable Effect and place it on the site for all to use.

Conceptually, the task of extending the JMF is relatively simple, although the programming task might not be trivial (for example, adding support for a new video codec). The process consists of three steps:

By registering the new classes with the JMF, they effectively become added to the pool of resources that the manager classes (chiefly Manager) oversee. The manager is then aware of the new class (and the features it provides) and can create an instance of that class as needed. Figure 9.9 shows the three-step process. A correctly registered class is subsequently available through the Manager class for any application.

Before specifics are discussed, it is worth mentioning that extending the JMF isn't a trivial undertaking. The complexity does vary depending on the task at hand. However, a good understanding of the lower-level structures of the JMF (for example, Buffer, Track, Format and so on) is a virtual prerequisite for any such task. Those without such an understanding will no doubt acquire it if their extensions are successful.

Role of Interfaces

With the exception of DataSource, all the key infrastructure functional classes of the JMF employed repeatedly in user programs—Player, Processor, DataSink, and all the PlugIns (Multiplexer, Demultiplexer, Codec, Effect, and Renderer)—are all interfaces. Extending the JMF by adding another type of Player (for instance) is then a case of writing a class that implements the Player interface.

Implementing an interface is a case of writing a class that possesses all the methods that the interface lists. All methods must be present and must possess exactly the same signature as that in the interface. The signature of a method is its name, return type, visibility, argument numbers, type, and ordering.

No inheritance is involved in implementing an interface, so all code must be written by the user—there are no default versions for each method. On the other hand, the user class is free to extend another class (class, not interface) and thus inherit any methods and attributes it provides.

For instance an implementation of the Codec interface would have to possess 11 methods—five directly from the Codec interface along with four from the PlugIn interface that Codec extends, and two from the Controls interface that Codec also extends. Listing 9.6 shows an empty class that implements the Codec interface. It possesses all the necessary methods, but they are all stubs (empty methods that perform no useful task). As such, it is the bare skeleton of a real Codec implementation—it currently does nothing.

Listing 9.6 An Empty Class that Implements the Codec Interface

import javax.media.*;

/*******************************************************
* A stub or empty class with no functionality but which
* shows the bare minimum methods a class must possess in
* order to extend the Codec interface (to be a Codec).
*
* @author Spike Barlow
********************************************************/
public class EmptyCodec implements Codec {

/////////////////////////////
// 5 methods of Codec itself
/////////////////////////////
public Format[] getSupportedInputFormats() { return null; }

public Format[] getSupportedOutputFormats(Format in) { return null; }

public int process(Buffer in, Buffer out) {
  return PlugIn.INPUT_BUFFER_NOT_CONSUMED;
}

public Format setInputFormat(Format in) { return null; }

public Format setOutputFormat(Format out) { return null; }

//////////////////////////////////////////////
// The Controls interface that Codec extends
/////////////////////////////////////////////
public Object[] getControls() { return null; }

public Object getControl(String name) { return null; }

///////////////////////////////////////////
// The PlugIn interface that Codec extends
///////////////////////////////////////////
public void close() { }

public String getName() { return "EmptyCodec"; }

public void open() { }

public void reset() { }
}

Two minor catches exist for the unwary in implementing an interface. The first is that many interfaces are tiered, extending one or more other interfaces. If such an interface is being implemented, not only must its own methods be provided (written), but also all methods of interfaces that the interface in question extends. For instance, the Codec interface extends both Controls and PlugIn interfaces. Thus any class that implements Codec must possess at least 11 methods: five of Codec itself, four of PlugIn, and two of Controls. Second, the visibility of methods within the current version of the JMF API documentation often appear as though they are a package (that is, the method has no visibility modifier), whereas they are in fact public methods. Fixing these cases are simple however because the compiler will raise an error indicating that the interface specifies public visibility.

Registering the New Classes

For a new class such as a new Player to be automatically available for subsequent use, it must be registered with the JMF. In effect this registration process can be thought of as adding the new class to the database of classes that the JMF maintains. After that addition is made, the JMF is aware of the class. From then on, JMF will create instances of the class as appropriate (in response to user requirements).

Several of the manager classes have a role in registration of new classes:

PlugInManager— New PlugIns (Codecs, Renderers, and so on) are added (registered) via the PlugInManager. They can also be removed via this class.

PackageManager— New MediaHandlers—Players, Processors, DataSource, and DataSink classes—are registered via this class.

CaptureDeviceManager— Newly attached capture devices are registered with the JMF via this class.

Registering a new class is a somewhat finicky process involving precise naming rules for the class (if a MediaHandler). It is also an uncommon process. Rather than writing code to register a new class, which is possible by making calls to the appropriate manager class, a new class can be registered interactively with the JMFRegistry program.

JMFRegistry is a GUI application that comes as part of the standard JMF distribution. Using menus, buttons, and text fields, it allows the user to interactively add or delete PlugIns, CaptureDevices, and packages to or from the database that the local implementation of the JMF maintains. Using JMFRegistry is simple (provided the user comprehends the JMF rules for naming) and is strongly recommended as an easier means for registering a new class. Sun maintains a short help document for JMFRegistry. It can be found at http://java.sun.com/products/java-media/jmf/2.1.1/jmfregistry/jmfregistry.html. Running JMFRegistry is as simple as java JMFRegistry.

The JMF employs naming rules as its means of keeping track of MediaHandler (Player, Processor, and DataSink) and DataSource classes, as well as knowing what they do. Classes are organized into packages, and the package name together with the classname indicates what the class does. The JMF employs these rules for all MediaHandler and DataSink classes that come as part of the JMF. If the same naming rules aren't followed for new MediaHandler classes, the Manager class will be unable to find the new class. Hence it will remain unavailable to the user.

These naming rules don't apply for PlugIns. Each PlugIn is added separately as a fully qualified classname.

However, for all DataHandler and DataSource classes, strict naming rules apply. All DataHandler classes (whether Player, Processor, or DataSink) must be called (has a classname of) Handler, whereas all DataSources must be called (have a classname of) DataSource. This apparent confusion stemming from a profusion of Handler and DataSource classes is resolved by each of them existing in their own package. Hence the fully qualified name of the class uniquely differentiates it from all other classes.

Package names are split into several portions—an initial user assigned name followed by a fixed portion that reflects the type of class it is (Processor, Player, DataSink, and so on), followed by a name that reflects the protocol or content-type that the class supports. That is followed by the classname.

In particular,

• Players are named as <content package-prefix>.media.content.<content-type>.Handler

• Processors are named as <content package-prefix>.media.processor.<content-type>.Handler

• DataSinks are named as <content package-prefix>.media.datasink.<protocol>.Handler

• DataSources are named as <protocol package-prefix>.media.protocol. <protocol>.DataSource

The names content package prefix and protocol package prefix are Sun's terms for the user assigned prefix. The JMF provides for two categories of prefixes: one for MediaHandlers (content package prefix) and one for DataSources (protocol package prefix). For instance the prefixes that are part of the JMF 2.1.1 (Windows Performance) distribution are javax, com.sun, and com.ibm.

So, for instance, com.sun.media.protocol.rtp.DataSource is the name of the DataSource class provided by Sun, as part of the JMF, for RTP DataSources. A user writing her own processor for handling a new content type known as, for example, cs9 and having selected a package prefix of au.edu.adfa, for instance, would name the class au.edu.adfa.media.processor.cs9.Handler.java.

Implementing PlugIns

Implementing a PlugIn is generally easier than writing a DataSource or DataHandler. Not only is the task generally smaller (which isn't always so), it doesn't require the strict class and package naming needed for DataSources and DataHandlers.

The discussion under the previous section concerning interfaces and Listing 9.6 provides an example of how implementing a Codec might be started. Similar approaches apply for Demultiplexers, Effects, Multiplexers, and Renderers.

Registering a new PlugIn with the JMF (either using JMFRegistry or PlugInManager directly) means that the PlugIn will henceforth be available to default Processors or those created with a ProcessorModel object.

It is possible to use a nonregistered PlugIn by creating an instance of the class directly and using the Processor object's TrackControl objects (one per track composing the media) to specify that the PlugIn be employed. This might, for instance, be used as a means of testing a PlugIn under development. However, in general, registering a PlugIn is far preferable.

Implementing DataHandlers

Implementing a new DataHandler—a Player, Processor, or DataSink—requires writing a class named Handler that is part of a larger package and which implements the particular interface (Player, Processor, or DataSink).

As for PlugIn implementation, implementing a DataHandler requires writing a class that possesses all the methods listed for that interface, as well as for all the interfaces it extends. This is a nontrivial task—a DataSink must possess at least 12 methods, a Player must have 32 methods, and a Processor must have 38. The complexity of Player stems from the fact that it extends Controller, which extends Clock. Processor extends Player and hence has an additional six methods.

As discussed in the previous section on registering new classes, a new MediaHandler class must be called Handler and exist in a package with a particular name structure. The exact naming rules are found previously, whereas the following code fragment shows the start of a Player class that handles a hypothetical content-type known as 4XXXX.

package com.samspublishing.mediaaips.media.content.4XXXX
import javax.media.*;
public class Handler implements Player {
:        :        :
}

Note that the package to which the class belongs begins with the top-level name. In this case, the hypothetical com.samspublishing.mediaapis, has the mandatory media.content as the mid-portion of the name, indicating it is a player, and 4XXXX as the suffix, indicating the particular content-type that it handles.

It is worth noting that individual Handlers aren't registered with the JMF, but simply the top-level, user package name. After that is registered, all subsequently implemented MediaHandlers in the same top-level package will be automatically found by the Manager class when it is asked to create a new object.

Extending DataSource

The DataSource class is an exception in terms of extending the JMF. Whereas PlugIns and DataHandlers are interfaces and require one pattern for implementation, DataSource is a class and requires a different approach.

Writing a new DataSource means extending the existing DataSource class or one of its subclasses. When contrasted with implementing an interface, this can be an advantage because the default inherited behavior for some methods might not need to be altered. The following code fragment shows the start of a DataSource class that deals with a new protocol known as sdtb, which is a type of pull data source.

package com.samspublishing.mediaapis.media.protocol.sdtb
import javax.media.*;
import javax.media.protocol.*;
public class DataSource extends PullBufferDataSource {
:        :        :
}

As for MediaHandlers, the top-level, user-supplied, package name must be registered with the JMF for the new DataSource to be accessible to the Manager class. This is most easily done through the JMFRegistry application. The JMF differentiates user-package names into those for DataHandlers, which it names content package prefix, and those for DataSources, which it calls protocol package prefix. Registration of each package prefix is separate even, as is the usual case, if the names are the same.

Sun's Examples

As mentioned previously, extending the JMF is a non-trivial task and often involves writing a relatively large class. This section doesn't possess a complete example because of space and complexity restrictions.

However, Sun has provided a number of excellent examples of writing PlugIns, MediaHandlers, and DataSources in both its JMF guide http://java.sun.com/products/java-media/jmf/2.1.1/guide/JMFTOC.html and on its solutions Web page http://java.sun.com/products/java-media/jmf/2.1.1/solutions/index.html. These are very good starting points for those wanting to extend the JMF and go beyond the information provided here. They also provide a good gauge for the complexity of the task.

In particular, the current version of the JMF guide contains a Demultiplexer PlugIn implementation for GSM, a gain control, a Renderer for RGB (employing AWT's Image class), an ftp DataSource, and a Controller for a hypothetical type of data known as TimeLine.

The solutions page contains a particularly clever implementation of a DataSource that provides screen capture facilities. By defining a new protocol called screen, the DataSource can be used to capture a particular region of the computer screen as a video. The implementation is highly recommended not only as an example of what can be done with the JMF, but also is a particularly useful utility—one that can be installed within the JMF in minutes.