Time Trigger

Time-based triggers are easy to explain and resembles the Unix cron utility. In a time-aware coordinator, a workflow is executed at fixed intervals or frequency. A user typically specifies a time trigger in the coordinator using three attributes:

Start time (ST)

Determines when to execute the first instance of the workflow

Frequency (F)

Specifies the interval for the subsequent executions

End time (ET)

Bounds the last execution start time (i.e., no new execution is permitted on or after this time)

In other words, the first execution occurs at the ST and subsequent executions occur at (ST + F), (ST + 2F), (ST + 3F) , and so on until the ET is reached.

Data Availability Trigger

Workflow jobs usually process some input data and produce new output data. Therefore, it is very common to hold off the workflow execution until all of the required input data becomes available. For instance, you want to execute a workflow at 1 a.m., but you also want to make sure the required input data is available before the workflow starts. You ideally want the job to wait even past 1 p.m. if any of the input data is missing.

The Oozie coordinator supports a very flexible data dependency–based triggering framework. It is important to note that the concept of data availability–based scheduling is a little more involved than time-based triggering. Therefore, we introduce the concept here but explain data triggers in detail in Chapter 7.

Coordinator Action

A coordinator job regularly creates/materializes a new coordinator action for each time instance based on its start time and frequency. For example, if a coordinator job has a start time of January 1, 2014, and an end time of December 31, 2014, with a frequency of one day, there will be a total of 365 actions, one created each day. More importantly, the coordinator action actually checks for data availability and ultimately submits the workflow.

Our First Coordinator Job

In this section, we describe a basic coordinator job that is very similar to a Unix cron job. This example introduces the common terminologies and concepts of a time-triggered coordinator job. As mentioned earlier, the time-triggered coordinator launches the workflow starting from the start time and continuously launches one at every predefined interval (a.k.a. frequency) until it reaches the end time. In this example, we want to execute the identity-WF (explained in Chapter 1) daily starting from 2 a.m., January 1, 2014 to 2 a.m., December 31, 2014. That means the first coordinator action will start at 2014-01-01T02:00Z, the second instance will start at 2014-01-02T02:00Z, and the last instance at 2014-12-31T02:00Z. Each of these time instances is called the nominal time of that specific action. In other words, each coordinator action must have a nominal time. Here’s the formal XML definition of such a coordinator:

<coordinator-app name="my_first_coord_job" start="2014-01-01T02:00Z " 
      end=“2014-12-31T02:00Z" frequency="1440"  timezone="UTC" 
      xmlns="uri:oozie:coordinator:0.4">
   <action>
      <workflow>
         <app-path>${appBaseDir}/app/</app-path>
         <configuration>
         <property>
            <name>nameNode</name>
            <value>${nameNode}</value>
          </property>
          <property>
            <name>jobTracker</name>
            <value>${jobTracker}</value>
          </property>
          <property>
            <name>exampleDir</name>
            <value>${appBaseDir}</value>
          </property>
         </configuration>
       </workflow>
   </action>
</coordinator-app>

For ease of explanation, we artificially divide the above coordinator XML into two segments: specification of the trigger(s) and definition of the triggered workflow. The first segment primarily describes the triggering conditions including both time and data dependencies.

As mentioned earlier, there are three main attributes required to specify a coordinator job. The start time defines when to start the execution and the value could be some time in the future or some time from the past. The end time defines the time when a coordinator should stop the creation of new coordinator actions. Both start and end times are defined in a combined date and time format as defined by ISO 8601. The frequency of the job is 1,440 minutes or one day. Although the default unit of coordinator frequency is minutes, there are other convenient ways to specify the frequency using EL functions that we describe in “Parameterization of the Coordinator”. More specifically, for daily jobs, we recommend you use ${coord:days(1)} instead of 1,440 minutes for frequency.

In addition, there are two other self-explanatory attributes that are not directly related to the triggering mechanism. The first attribute is name with value my_first_coord_job that can later be used for querying Oozie. The second attribute is xmlns, which specifies the coordinator namespace used for coordinator XML versioning. The namespace plays a critical role in ensuring backward compatibility of the coordinator. For example, the new/updated features added for namespace oozie:coordinator:0.4 might break or modify the functionality of a coordinator written with an older namespace (e.g., oozie:coordinator:0.3). Alternatively, if you want to use some of the new/updated features, the new namespace should be used.

The next segment of the coordinator XML specifies what type of job to execute when the triggering conditions are met. As of now, Oozie coordinator only supports launching Oozie workflows and a coordinator application can only include one workflow application. In the future, the scope could be extended to other types of jobs as well.

The workflow tag here is the same as the one we used to define a standalone submission in “A Simple Oozie Job”. coordinator uses these values to parameterize and automate the workflow submission as we do with any standalone workflow submission. The main difference is in the representation. In a standalone workflow execution, we typically use a property file in key-value format, though the XML syntax is also supported. But for the workflow execution via a coordinator, we have to define it inline in XML format. In both cases, these key-value pairs are passed to the workflow at its start.

It is important to note that the propagation of coordinator properties down to the workflow is not automatic; you need to define and specify these key value pairs under <action> configuration. As shown in the example, we specify app-path to point to the workflow application path, whereas in a standalone CLI-based workflow submission we define the property oozie.wf.application.path in the property file to specify the same thing. The rest of the parameters are optional and defined as configuration properties. The properties defined in this example are similar to the properties defined in “A Simple Oozie Job”.

Coordinator Submission

There are multiple ways to submit a coordinator. In this section, we only explain job submission using the oozie CLI. Other approaches are described later in Chapter 11. At first, we will need to create a local properties file (say job.properties) and pass this filename during submission as an argument to the CLI:

$ cat job.properties
nameNode=hdfs://localhost:8020
jobTracker=localhost:8032
appBaseDir=${nameNode}/user/${user.name}/ch06-first-coord
oozie.coord.application.path=${appBaseDir}/app

In addition, we will need to upload the coordinator job definition to HDFS. It primarily includes the job’s XML definition (i.e., coordinator.xml):

$ hdfs dfs -put ch06-first-coord/ .
$ hdfs dfs -ls -R ch06-first-coord/
drwxr-xr-x   - joe supergroup       0 2014-03-29 12:24 ch06-first-coord/app
-rw-r--r--   1 joe supergroup     705 2014-03-29 12:24 ch06-first-coord/app/
coordinator.xml
-rw-r--r--   1 joe supergroup    2141 2014-03-29 12:24 ch06-first-coord/app/
workflow.xml
drwxr-xr-x   - joe supergroup       0 2014-03-29 12:24 ch06-first-coord/data
drwxr-xr-x   - joe supergroup       0 2014-03-29 12:24 ch06-first-coord/data/
input
-rw-r--r--   1 joe supergroup      25 2014-03-29 12:24 ch06-first-coord/data/
input/input.txt

The command in Example 6-1 submits the coordinator and returns a coordinator job ID if successful. The subsequent commands show the most common operations that users typically run for monitoring and managing the coordinator jobs through the Oozie CLI.

$ export OOZIE_URL=http://localhost:11000/oozie
$ oozie   job -run -config  job.properties 
job: 0000003-140329120933279-oozie-joe-C
$ oozie job -info 0000003-140329120933279-oozie-joe-C 
Job ID : 0000003-140329120933279-oozie-joe-C
--------------------------------------------------------------------------------
Job Name    : my_first_coord_job
App Path    : hdfs://localhost:8020/user/joe/ch06-first-coord/app
Status      : RUNNING
Start Time  : 2014-01-01 02:00 GMT
End Time    : 2014-12-31 02:00 GMT
Pause Time  : -
Concurrency : 1
--------------------------------------------------------------------------------
ID                                         Status    Ext ID                               
Err Code  Created              Nominal Time         
0000003-140329120933279-oozie-joe-C@1     SUCCEEDED 0000004-140329120933279-
  oozie-joe-W 
-         2014-03-29 23:14 GMT 2014-01-01 02:00 GMT 
--------------------------------------------------------------------------------
0000003-140329120933279-oozie-joe-C@2     SUCCEEDED 0000005-140329120933279-
  oozie-joe-W 
-         2014-03-29 23:16 GMT 2014-01-02 02:00 GMT 
--------------------------------------------------------------------------------

$ oozie job -info 0000003-140329120933279-oozie-joe-C@1 
ID : 0000003-140329120933279-oozie-joe-C@1
--------------------------------------------------------------------------------
Action Number        : 1
Console URL          : -
Error Code           : -
Error Message        : -
External ID          : 0000004-140329120933279-oozie-joe-W
External Status      : -
Job ID               : 0000003-140329120933279-oozie-joe-C
Tracker URI          : -
Created              : 2014-03-29 23:14 GMT
Nominal Time         : 2014-01-01 02:00 GMT
Status               : SUCCEEDED
Last Modified        : 2014-03-29 23:15 GMT
First Missing Dependency : -
--------------------------------------------------------------------------------

$ oozie job -kill 0000003-140329120933279-oozie-joe-C
$ oozie job -info 0000003-140329120933279-oozie-joe-C
Job ID : 0000003-140329120933279-oozie-joe-C
--------------------------------------------------------------------------------
Job Name    : my_first_coord_job
App Path    : hdfs://localhost:8020/user/joe/ch06-first-coord/app
Status      : KILLED
Start Time  : 2014-01-01 02:00 GMT
End Time    : 2014-12-31 02:00 GMT
Pause Time  : -
Concurrency : 1
--------------------------------------------------------------------------------
ID                                         Status    Ext ID                               
Err Code  Created              Nominal Time         
0000003-140329120933279-oozie-joe-C@1     SUCCEEDED 0000004-140329120933279-
  oozie-joe-W 
-         2014-03-29 23:14 GMT 2014-01-01 02:00 GMT 
--------------------------------------------------------------------------------
0000003-140329120933279-oozie-joe-C@2     SUCCEEDED 0000005-140329120933279-
  oozie-joe-W 
-         2014-03-29 23:16 GMT 2014-01-02 02:00 GMT 
--------------------------------------------------------------------------------

Oozie Web Interface for Coordinator Jobs

Oozie provides a basic, read-only user interface for coordinator jobs very similar to what it provides for workflows and bundles. Users can click on the Coordinator Jobs tab on the Oozie web interface at any time. It displays the list of recent coordinator jobs in a grid-like UI as shown in Figure 6-1. This UI captures most of the useful information about the coordinator jobs. The last column titled Next Materialization shows the nominal time for the next coordinator action to be materialized for any running coordinator job.

Figure 6-1. Oozie web interface for coordinator jobs

Users can drill down into a specific coordinator job by clicking on the row of that job. This will display a new window presenting the details of that coordinator job, as shown in Figure 6-2. As you can see, there are four tabs: Coord Job Info, Coord Job Definition, Coord Job Configuration, and Coord Job Log. You can select any of these tabs as necessary (the Coord Job Info tab is displayed by default). The first tab shows the current job status, including all the spawned coordinator actions listed in the bottom half of the window. Users can click on the reload icon located at the top-left of the window to refresh the contents. The second tab, Coord Job Definition, displays the original coordinator XML that you submitted.

All of the configuration settings passed as part of the CLI and the properties file are displayed in the third tab (the Coord Job Configuration tab). The fourth tab shows the Oozie log generated for this specific coordinator job. Since coordinator jobs typically create a lot of coordinator actions, they tend to be long running and the logs may be huge. It can take a long time to load all the logs from the Oozie backend. That’s why it is better to retrieve only the logs for a subset of coordinator actions. For this purpose, there is a “Retrieve log” button where the user can specify a set of coordinator action numbers such as “1,2” or “1-3,” and so on. This will ensure that Oozie retrieves the logs only for those actions.

Figure 6-2. Oozie web interface for coordinator jobs

From the first tab (Coord Job Info) of the coordinator job window, users can further drill down to the corresponding workflow by clicking on the row of the coordinator action. This displays the workflow window shown in Figure 6-3. This window is essentially the same as the one explained in “A Simple Oozie Job”.

Figure 6-3. A workflow job launched by coordinator

EL Functions for Frequency

In “Our First Coordinator Job”, we used frequency="1440" for daily jobs. This frequency was expressed in minutes. However, there are some scenarios where frequencies can’t be easily expressed in absolute minutes. For example, a frequency of one day may not always translate to 24 hours. Some days could be 23 hours or 25 hours due to Daylight Saving Time. Similarly, every month does not correspond to 30 days. It could be anything between 28 and 31 days. To help you handle these intricacies easily, Oozie provides a set of functions to define the frequency. We strongly encourage you to utilize those functions instead of using absolute value in minutes.

In a nutshell, the advantages of using Oozie-defined EL functions include:

• Transparent handling of Daylight Saving Time

• Makes the application portable across time zones

• Easy handling of Daylight Saving Time or any time-related policy changes in various countries

Day-Based Frequency

Oozie provides two EL functions to specify day-based frequencies. The first one, coord:days(N), means the number of minutes in N days. The second one, coord:endOfDays(N), means the same thing as coord:days(N). The only difference is that endOfDays shifts the first occurrence to the end of the day and then adds N days to get the next occurrence. Table 6-1 explains the different scenarios.

Month-Based Frequency

Any month-based frequency also has its own issues similar to a day-based frequency. These include:

• Number of days in a month is not a constant, but changes month to month. It also depends on whether the year is a leap year or not.

• Number of hours in the individual days of a month might not be the same due to Daylight Saving Time.

There are two month-based EL functions for frequency. coord:months(N) returns the number of minutes in N months starting from the current nominal time. coord:endOfMonths(N) is very similar to coord:months(N). The difference is that endOfMonths() first moves the current nominal time to the end of this month and then calculates the number of minutes for N months from that point. Table 6-2 demonstrates the various scenarios with real values.