Detailed Time Model

As discussed in Chapter 7, the JMF employs a layered approach to its representation of time. At the low-level end of the time model are classes for representing time to nanosecond accuracy. At the high-level end of the model, the JMF sees controllers as being in one of a number of states that are transitioned between under program control.

Low-Level Time: Time and TimeBase Classes

Two classes, Time and SystemTimeBase, and one interface, TimeBase, detail the JMF's low-level model of time.

At the bottom of the hierarchy and also perhaps the most fundamental is the Time class. A Time object represents a particular instant in time to one nanosecond accuracy. Figure 8.3 shows the Time class's constructors and public methods.

Time objects are often returned by methods used to query the temporal status of another object; for instance, the amount of elapsed play time on some media.

Similarly, Time objects can be employed to alter the temporal properties of an object. For instance, to specify a particular point in time from which to start playback of media, a Time object might be employed as follows:

The Time class also possesses a special constant TIME_UNKNOWN. This constant finds applications in contexts in which an object might be asked the duration of the media it is associated with, but its length hasn't, or cannot, be ascertained.

The Time class specifies a single instant in time, whereas time-based media, by its nature, is dynamic and time varying. JMF's support for ticking (at 1 nanosecond per tick) time comes in the form of the TimeBase interface. The TimeBase interface is an important one, and one that is implemented by a number of important classes. (More accurately, it is subsumed in other key interfaces such as Controller and Player, which extend the interface.) The TimeBase interface defines only two methods, both of which are used to query the current time of the TimeBase object. The getTime() method returns a Time object. An alternate means of obtaining the same information is the getNanoseconds() method, which returns a long. There is no provision in a TimeBase for altering time: It can only be queried regarding its current state.

As a default implementation of the TimeBase interface, JMF provides the class SystemTimeBase. SystemTimeBase has a single empty constructor and only the two methods defined in the TimeBase interface. Alternatively, the system time base can be obtained through the Manager class's getSystemTimeBase() method.

The Clock Interface

Those classes that implement the TimeBase interface provide a constantly ticking, unalterable source of time. However, controlling media means providing control over the temporal properties of that media: being able to start or stop the media at arbitrary locations as well as control its rate (for example, fast forward or rewind on a player). The Clock interface is the means by which that is achieved, and it is implemented by objects that support the JMF model of time.

In many ways, the Clock interface is pivotal both as the cement between the low-level and high-level time models and as a core interface of the API. The Controller, Player, and Processor interfaces, all central to the functionality of the JMF, extend Clock.

Clocks are typically associated with a media object. Indeed, control over media such as playing or processing entails having a Clock associated with that media. (Because Controller, Player, and Processor interfaces extend Clock, the Player or Processor object is also the clock.) A Clock serves as both the timekeeper for its media and also a means of altering and adjusting the time of that media. The time a clock keeps is known as the media time.

Clocks achieve their dual task of monitoring and altering the time flow of their associated media by employing a TimeBase. As noted, TimeBase objects represent constantly ticking and unalterable time. Therefore, the Clock provides a remapping or transform from the TimeBase time to that associated with the media. This is a simple linear transform requiring three parameters: the rate (for example, of play), the media start time, and the time base start time. From these, the media time can be determined as follows:

media_time = media_start_time + rate x (time_base_time - time_base_start_time)

The meanings of the previous terms are as follows:

Media time— The media's own position in time. For instance, if an audio clip was one minute in length, its media time would range between 0 and 60 seconds.

Media start time— The offset within the media from which play is started. If play starts from the beginning of the media, this value is 0. If it was started seven and a half seconds in, this value would be 7.5.

Rate— The rate of time passage for the media. A rate of 1 represents normal forward passage (for example, play), whereas a value of -5 would represent a fast rewind.

Time base time— The time of the TimeBase object that the Clock incorporates. This starts ticking (increasing) as soon as the Clock object is created and never stops.

Time base start time— The time of the TimeBase object at which the Clock is started and synchronized with the TimeBase. For instance, the Clock might be started 3.2 seconds after the Clock was created (and hence the TimeBase was also created and started ticking). Hence, the time base start time would be 3.2 seconds.

A Clock is in one of two possible states: Started or Stopped. A clock is started by making the syncStart() method call. The syncStart() method accepts a single argument being the time base start time from which the Clock should be started. Once the Clock's TimeBase object reaches that time, the clock will synchronize with the TimeBase and enter the Started state. This mechanism allows a Clock to be set to start at some future time (or at the current time by passing the syncStart() method the Clock's own TimeBase object's current time). Any changes to the media (start) time and rate must be performed before a Clock enters the Started state. Attempting to use the methods that carry out these operations on a Clock in the Started state will result in a ClockStartedError being thrown. Thus, the usual steps in starting a clock are

A Clock's initial state is Stopped. After a clock is Started, it can be stopped in one of two ways. It can either be stopped immediately with the stop() method, or a media stop time can be set: Once the media time reaches (or if it has already exceeded) that time, the Clock will stop.

Finally, it is worth noting that it is possible to synchronize two or more Clocks by setting them to use the same TimeBase object. The Clock interface exposes methods for getting and setting the TimeBase object associated with the clock.

Figure 8.4 shows all the methods of the Clock interface. Besides those already discussed, often used methods are getMediaTime() and getMediaNanoseconds(). For instance, these might be called repeatedly as media is being played in order to provide some feedback on elapsed time for the viewer. Similarly, setRate() and setMediaTime() are used to provide user control in playback scenarios, but might only be called on a stopped clock.

High Level Time: The Controller Interface

The Controller interface directly extends Clock in three areas:

• Extends the concept of Stopped into a number of states concerning resource allocation, so that time-consuming process can be better tracked and controlled.

• Provides an event mechanism by which the states can be tracked.

• Provides a mechanism by which objects providing further control over the controller can be obtained.

It is on top of the interface that the commonly used Player—which is used in the last example in Chapter 7—and Processor interfaces sit.

As we explained in the previous chapter, achieving the state where the control, processing, or play of media can be started isn't an instantaneous operation. Resources need to be gathered in order to support that control. Tasks involved in resource gathering include opening files for reading, filling buffers, or gaining exclusive control of hardware devices (for example, a hardware decoder). This point is illustrated in the next subsection in which the time taken to gather resources so that a video can be played is shown.

The Controller interface subdivides the Stopped category of Clock into five stages that reflect the state of preparedness of the Controller: how close it is to being capable of being started. Those five states, in order of least prepared through prepared to start, are

Unrealized— A Controller that has been created but hasn't undertaken any resource gathering.

Realizing— A transition state reflecting the fact that the Controller is gathering information about the resources needed for its task, as well gathering resources themselves.

Realized— A steady state reflecting a Controller that has gathered all the nonexclusive resources needed for a task.

Prefetching— A transition state reflecting the fact that the Controller is gathering all resources needed for its task that weren't obtained in the realizing state. Typically this means acquiring exclusive usage resources such as hardware.

Prefetched— The Controller has acquired all necessary resources, performed all pre-startup processing, and is ready to be started.

The Controller interface provides program control for the movement between these states via a set of methods. Similarly, the Controller interface allows for program monitoring of those transitions via an event system. Objects can implement the ControllerListener interface and thus be sent events as the Controller transitions between the various states. Figure 8.5 shows the methods and associated transitions between states. Figure 8.6 shows the events that are generated as a Controller transitions between its states.

As shown in Figure 8.5, a Controller has five methods for controlling transition between states. The forward motion methods are realize(), prefetch(), and synchStart() (from the Clock interface) for moving the Controller into a more prepared, and finally Started, state. They are asynchronous—they return immediately, but the engendered action generally takes some time to complete. When the transition is complete (or interrupted by a stop() or other such call), an event is posted. The reverse direction methods are stop() and deallocate(). These are synchronous methods. stop() is used to stop a started Controller. The Controller transitions to Prefetched (or in some cases where resources must be relinquished: Realized) and might subsequently be restarted. deallocate() frees the resources consumed by a Controller and should be used for that purpose (for example, in the stop() method of an applet). deallocate() cannot be called on a Started Controller; it must be stopped first. deallocate() returns a Controller to the Realized state if it is in that state or greater (a state closer to Started); otherwise, the Controller returns to Unrealized.

A Controller posts events about its state changes. Those objects wanting to be informed about Controller events must implement the ControllerListener interface. The ControllerListener interface consists of a single method:

public synchronized void controllerUpdate(ControllerEvent e)

Objects communicate their desire to be sent a Controlleris events by calling that Controlleris addControllerListener() method.

The events posted by a Controller fall into one of four categories:

• Life cycle transitions

• Method acknowledgements

• Status change information

• Error notification

The TransitionEvent, or a subclass such as EndOfMediaEvent, is a Controller's means of reporting state changes. The method acknowledgement events RealizeCompleteEvent, PrefetchCompleteEvent, StartEvent, DeallocateEvent, and StopByRequestEvent are used to communicate the fulfillment of the corresponding methods—for example, realize()—called on the Controller. There are three status change events: RateChangeEvent, StopTimeChangeEvent, and MediaTimeSetEvent that inform the listener of changes in rate, stop time, and when a new media time is set. Errors fall under the ControllerErrorEvent class and include ResourceUnavailableEvent, DataLostErrorEvent, and InternalErrorEvent. Other errors are thrown as exceptions. For instance, attempting to syncStart() a Controller before it achieves the Prefetched state will result in a NotPrefetchedError being thrown.

Figure 8.7 shows all the methods and constants of Controller that aren't inherited from the Clock or Duration interfaces. Among the important methods of the interface not discussed previously are close(), getStartLatency(), getControl(), and getControls(). close() is used to release all resources and cease all activity associated with a Controller. The Controller can no longer be employed (its methods called) after it has been closed. The getStartLatency() method returns an estimate of the amount of time required in a worst-case scenario before the first frame of data will be presented. It is used to provide an estimate for syncStart() calls. The estimate is more accurate if the Controller is in the Prefetched state. The getControl() and getControls() methods provide a means for obtaining Control objects. These can be used to alter the behavior of the Controller. Controls and Controllers are two different things despite their unfortunate similarity in name. Controls are discussed in a subsequent section.

Timing a Player

In order to illustrate the concepts covered in this section and the inter-relationship between the high- and low-level models of time that the JMF supports, the simple player applet BBPApplet from the previous chapter was modified slightly.

The modification consisted of time-stamping and printing every Controller event that was received by the applet. This provides a map through time of the course of the player from the instant it is started, until the time the media being played (an mpeg video) finishes.

The modification itself is simple, and rather than reproducing the code of the entire applet again, only the changes made will be discussed. A SystemTimeBase object was constructed immediately prior to the Player being started. Each time the controllerUpdate() method was entered, the SystemTimeBase object was queried as to the time, and that value plus the event that was received were printed to the screen. The three steps were

// The object to be used to timestamp Controller events.
protected SystemTimeBase    timer;

timer = new SystemTimeBase();
player.start();

// Print the time the event was received, together with the event,
// to the screen.
System.out.println(""+(double)timer.getNanoseconds()/Time.ONE_SECOND
    + ": " + e);

The timing data output when the applet played an mpeg video somewhat over two minutes in length is reproduced in Listing 8.1. Each Controller event received by the class (generated by the Player) has been time stamped and output. The events occur in chronological order with a timestamp (in seconds) being the first piece of information on the line, that being followed by the event itself. Figure 8.8 shows the state transitions through time, distilling the most important information.

Listing 8.1 Timing information output by the modified BBPApplet (Bare Bones Player Applet)

0.06: javax.media.TransitionEvent[source=com.sun.media.content.video.mpeg.Handle
r@275d39,previous=Unrealized,current=Realizing,target=Started]

1.6: javax.media.DurationUpdateEvent[source=com.sun.media.content.video.
mpeg.Handler@275d39,duration=javax.media.Time@10fe28
1.87: javax.media.RealizeCompleteEvent[source=com.sun.media.content.video.mpeg.H
andler@275d39,previous=Realizing,current=Realized,target=Started]

2.59: javax.media.TransitionEvent[source=com.sun.media.content.video.mpeg.Handle
r@275d39,previous=Realized,current=Prefetching,target=Started]

2.59: javax.media.PrefetchCompleteEvent[source=com.sun.media.content.video.mpeg.
Handler@275d39,previous=Prefetching,current=Prefetched,target=Started]

2.59: javax.media.StartEvent[source=com.sun.media.content.video.mpeg.Handler@275
d39,previous=Prefetched,current=Started,target=Started,mediaTime=javax.media.Tim
e@36e39f,timeBaseTime=javax.media.Time@19dc16]

160.06: javax.media.EndOfMediaEvent[source=com.sun.media.content.video.mpeg.Hand
ler@275d39,previous=Started,current=Prefetched,target=Prefetched,mediaTime=javax
.media.Time@60a26f]

160.06: javax.media.DurationUpdateEvent[source=com.sun.media.content.video.mpeg.
Handler@275d39,duration=javax.media.Time@484a05

What is clear from the output and figure is that the steps in preparing to play media are lengthy, particularly from the perspective of a computer that executes millions of instructions per second. The realizing step took over a second and a half, whereas the prefetch step required nearly three quarters of a second, all for media stored locally on the hard disk.