Today companies are using the Web to create new and better ways of serving their customers. Competition has become severe and is pushing the corporations to adopt new and different kinds of partnerships with suppliers, customers, and in some cases, even their own competitors.
Many electronic vertical exchanges provide services in which companies (buyers) can post their requirements and vendors or suppliers can then bid and compete for the business. Also, the companies must integrate their different units to serve customers better and gain competitive advantages. These exchanges require complex business arrangements in dynamic business processes and data being processed between the exchange, bidders, buyers, and different units. Businesses have to carefully construct their information systems so that they integrate well with their collaboraters and also continue to work with their own legacy systems. Data integration has become important with the increasing need for businesses to share information within and outside their organizations to reduce expensive inventories and costs.
There is a need for integration solutions that can provide standards-based functionality for data integration and business-to-business integration.
Data integration is a process by which data from disparate systems can be merged and used uniformly by transforming it in a universally acceptable and recognizable format.
Modern enterprises have a wide variety of applications interacting with each other using different message formats (see Figure 36.1). The data formats can be of different kinds, such as binary, XML, plain-text, and delimited text. WebLogic Integration helps applications to exchange data by supporting data translation and transformation. XML has become the standard for exchanging information between applications.
Some of the important terms in data integration are explained in the following sections.
Translation is conversion of data from one format (or language) to another format (or language). Binary data originating from a legacy system is typically converted to XML. For the translation process, each field of the binary data is converted to XML according to metadata defined for that field. The metadata typically includes the name of the field, the data type, the size, and a flag to indicate whether the data is optional or required. Figure 36.2 shows an example of data translation of some sample legacy data into XML.
Transformation is a process that leads to the alteration of a given data object. For example, one XML document is “transformed” to another XML document using Extensible Stylesheet Language Transformation (XSLT). This process of transformation using XSLT requires identifying, copying, and manipulating data elements from one XML format to another. When XSLT is used, the Extensible Stylesheet Language (XSL) is used to describe a series of transformations that can be performed on a given XML document. Figure 36.3 shows the translated data (as shown in Figure 36.2) being transformed into another XML document.
WebLogic Integration enables data integration by supporting the translation of binary legacy system data to XML and vice versa. After the data is converted into XML, it can be easily transformed into a specific XML grammar and can then be used to trigger WebLogic Integration workflows. Within the workflows, the data can be transformed to different formats and passed onto different subsystems for further processing. WebLogic Integration provides various components to support the translation from non-XML data into XML and vice versa. These components or functional areas are
Design-time component
Runtime component
Plug-in to BPM
The design-time component of WLI, which facilitates data integration, is an application called Format Builder. Format Builder helps in creating descriptions and input formats that match inbound data (such as binary data records) that needs to be converted. A description is typically composed of the layout and hierarchy of various data elements in binary data. When you use Format Builder, the description of the data is saved in an XML grammar called Message Format Language (MFL). This grammar describes the metadata of the binary data to be converted and is used by the runtime component and plug-in to BPM to translate binary data to an XML document or vice versa. Format Builder also creates a DTD or XML Schema for the XML document created from a translation for validation purposes. Format Builder will be discussed in detail later in this chapter.
The runtime component of WLI provides functionality that can be used to translate data from a binary format to XML and vice versa. This runtime component can be used in many ways; for example, it can be used as an Enterprise JavaBean (EJB) or as a business operation from a workflow in Studio. Figure 36.4 illustrates the runtime process flow.
The Business Process Management (BPM) component of WLI provides functionality to design, execute, and monitor complex business processes using a process engine. As described in Chapter 34, “Business and Workflow Process Management Using WLI,” BPM supports a plug-in framework to extend its basic functionality by modifying the behavior of components, including Start nodes, Event nodes, and Task actions. WLI includes a data integration plug-in that provides translation capabilities (XML to binary and vice versa) through a Task action. This functionality helps support an exchange of information between applications by making it possible to translate binary data from legacy systems into XML. This plug-in also provides the capability for event data processing in binary format and in-memory caching of MFL documents and translation object pooling for performance.
XML has become the standard for universal formatting of information and therefore is used as a standard for data integration in WLI. XML is used for both messaging and data storage. Various tools provided in WLI help in processing XML documents: for creating XML documents within a workflow, storing and retrieving XML documents, searching XML documents, and transforming and translating data. Some of these tools will be covered in the following sections.
The XML Instance Editor is a tool that is used to construct well-formed XML documents within a workflow. Typically, an XML document is populated within a workflow with content of workflow variables. Also, element/attribute values may sometimes be extracted from an XML document and assigned to a workflow variable. The Instance Editor facilitates developing this kind of functionality. The XML Instance Editor is accessible via various action dialogs in Studio, such as
Set Workflow Variable (see Figure 36.5)
Post XML Event
Send XML to Client
Invoke Exception Handler
The Instance Editor also provides functionality to import or export XML documents from or to the file system or an XML repository.
The XML repository stores XML documents and related information for WLI, thereby promoting reuse of XML-related components across workflows. Contents of the XML repository may be exported or imported. The following types of entities can be stored in the repository:
Document type definition (DTD)
MFL files
XML Schema (XSD)
Text files
XML documents
XSLT template documents
Each entity stored in the XML repository, shown in Figure 36.6, must have a unique name. All the standard CRUD (Create, Read, Update, and Delete) functions are supported.
The XPath Wizard assists in generating XPath expressions to locate textual data, elements, and attributes in an XML document. The XPath Wizard, shown in Figure 36.7, is accessed from the Expression Builder tool.
The Format Builder is a tool used to assist in defining formats for inbound and outbound data. The formats can be created, viewed, and updated using this tool. The Format Builder by itself does not require a running instance of the WLI Server. However, the WLI Server needs to be running if the formats created are to be stored in the WLI repository. Before creating a format, you need to determine the following information:
Definition of binary field data (delimiters, length, and so on)
Names of fields in XML
Organization of fields
Occurrences of repeating groups
Consider an example in which you use Format Builder to convert binary data to XML. The binary data file to be used in this example is located at examples1person.data.
Manish|Devgan|Boulder|Colorado|USA|
The MFL file that is generated by Format Builder is located at examples1person.mfl.
To define a format for the conversion, start the Format Builder tool by following these steps:
Define a root element called
Person.Add a field to the root element by choosing Insert, Field, As Child (as shown in Figure 36.8).
Define the XML Field Name (
FirstName), data type (String), and a delimiter (|) and then click Apply.Repeat step 3 and define other fields—
LastName,County,State, andCountry.Define a field called
CarriageReturnLineFeedof typeLiteraland define the literal value as
To run and test the example, choose Tools, Test. When the Test window is up, open the binary data file by choosing File, Open Binary and then run the sample by choosing Translate, Binary to XML (see Figure 36.10).
BEA WebLogic Integration bundles Contivo Analyst, a design-time graphical mapping tool that can be used to generate XSL stylesheets. The WebLogic Integration runtime can execute these stylesheets to transform a source message to a target message. For more information on Contivo products refer to http://www.contivo.com.
When you create an MFL using Format Builder, the transformation between XML to and from binary data can be performed from within a BPM workflow. Two actions are provided in the BPM Action List in Studio:
XML to Binary
Binary to XML
As you learned in Chapter 34, a workflow can be started or triggered by JMS messages on the WLI BPM event queue. Messages placed on the JMS queue can be in any format. A binary message on the JMS queue can be transformed by the Data Integration plug-in into XML, which can trigger a workflow.
Note
▸ To learn how a workflow can be triggered, see “Business and Workflow Process Management Using WLI,” p. 1251.
Refer to Figure 36.11. Here, a binary message triggers a workflow.
Business-to-business integrations (B2Bi) are becoming common as business processes need to span enterprise boundaries to meet the demand of today’s competitive environment, where improving business efficiency is the key to being successful. To support these processes, organizations need to support basic requirements such as workflow management, reliable messaging and routing, and application integration.
B2B integrations are becoming more real-time and are no longer being handled in batch mode. The data exchange protocols being followed, such as HTTP, FTP, and SOAP, are more standardized. B2B integration frameworks provide functionality to businesses to find and interact with trading partners electronically and securely, hence reducing costs. A trading partner is usually an external enterprise collaborating for the purposes of a business process.
WebLogic Integration provides a standards-based architecture for supporting B2B integration. Business protocols such as RosettaNet and ebXML, which provide the capability to exchange business documents in a secure manner with trading partners, are supported. A framework for managing execution and interaction of trading partner applications is also provided. The XML data repository and a set of configuration tools to manage the metadata and conversation definitions required for B2B integration are provided. Figure 36.12 shows the overall architecture and illustrates how B2Bi fits in with the other WebLogic Integration functional components.
WebLogic B2Bi functionality supports multiple deployment architectures. The most important architectures are described in the following two sections.
A WebLogic Integration solution can be configured to communicate directly with others in a peer-to-peer mode. This kind of deployment is desirable in a deployment in which, for example, a direct interaction between trading partners is required.
Figure 36.13 shows a peer-to-peer configuration. Here, customer A is interacting directly with suppliers B, C, and D. Suppliers B and C are running WebLogic Integration, and supplier D is running a third-party integration solution.
A hub-and-spoke configuration allows integration between two or more WebLogic Integration deployments via an intermediary. An intermediary (the hub) performs tasks such as routing and filtering messages, or it can provide services to the trading partners in the conversation.
Figure 36.14 illustrates a hub-and-spoke configuration. Here, a business is serving as auctioneer for business-to-business auctions. The business (represented by A) acts as an intermediary to buyers (B and C) and sellers (D and E).
A business message is a basic unit of communication among two business partners. It is important for a business message to abide by a protocol that is understood by all business partners involved. A business protocol usually specifies the structure of a business message. It may also specify characteristics of the message in terms of persistence, process of exchanging messages, and reliability. Some of the common business protocols are
Protocols such as RosettaNet and ebXML are supported by WebLogic Integration. For example, WebLogic Integration supports the creation and execution of workflows that model ebXML messages. Therefore, a business that deploys WebLogic Integration can use ebXML to interoperate with another business that deploys WLI’s B2Bi component or instead can have a lightweight trading partner solution called Business Connect, which will be discussed later in this chapter. By supporting the standard protocols, WLI facilitates seamless integration between business/trading partners. The out-of-the-box support for these standard protocols can be extended and customized as per the user’s need. This is achieved by a framework which is comprised of a set of Java classes which intercept and process business messages at runtime.
Figure 36.15 shows the structure of a sample RosettaNet business message based on the RosettaNet 1.1 protocol specification. A RosettaNet message is made up of three parts: header information containing the version number, the message body, and footer information. The message body is composed of several individual messages—namely, the preamble header, service header, and service content—which are constructed and then encoded as a MIME message.
Businesses exchange information by communicating via business messages. These messages form a part of a conversation. A conversation is a series of business messages exchanged between business partners. In a conversation, the participants play a role. The business messages that can be exchanged in a conversation between participants are determined by these roles. In WLI, each conversation has a unique name and version.
To start, a conversation definition is created; it consists of
A unique name and version.
A business protocol (such as ebXML).
Two or more roles to be used by business partners involved in the conversation.
Reference to a BPM workflow template for each role. BPM workflow templates are a recommended means of composing and exchanging business messages.
A very important aspect of B2B is the establishment and management of agreements among business partners. In WLI, this type of agreement is called a Collaboration Agreement. Using these colloboration agreements, trading or business partners agree on interactions in which they participate.
In B2B, trading partners participate in a conversation in a particular role. A workflow has to be implemented for each such role. These workflows, which encapsulate the processes required for a given trading partner’s role in a conversation, are known as collaborative workflows. Figure 34.5 (Chapter 34) shows collaborative workflows used in the end-to-end sample shipped with Platform 7.0. Here, based on the collaboration agreement between the E2E_Buyer and E2E_Hub trading partners, the E2E_BuyerQPAPublic workflow sends an XOCP message to the E2E_Hub trading partner. The E2E_Hub trading partner routes the QPA request to two other trading partners—E2E_SupplierOne and E2E_SupplierTwo—based on their respective collaboration agreements. Each supplier’s private workflow creates its own QPA response, which is eventually routed back to the buyer via the E2E_Hub, which acts as a “proxy supplier.”
Collaborative workflows are always linked with a particular conversation definition. While you’re defining a workflow template in Studio, you can use the B2B Integration tab in the Template Definition dialog box to define various aspects of the conversation definition, including name, version, role, and protocol (see Figure 36.16).
Consider an example in which there are two trading partners: Foo and Bar. Foo initiates a request and prepares an ebXML message and sends it to Bar. This process takes place under a defined collaboration agreement. To run this sample, follow these steps:
Start the WLI samples server by running
<wl_home>weblogic700samplesintegrationsamplesinRunSamples.cmdOpen Studio.
Log in as
admin/security.Use the Import package tool to import a given workflow under
chapter36example2FooBarWorkflows.jarin the ORG1 organization. FooBarWorkflows.jar contains both Foo and Bar workflows. Figure 36.17 shows the ask properties of “Send EBXML Message” of Foo Workflow. Figure 36.18 shows the start properties of the Start node of the Bar Workflow. This specifies that the workflow will trigger on an event caused by an ebXML message.Run BulkLoader to load the conversation definition, collaboration agreement, and other required definitions from
chapter36example2MyBulkLoaderData.xml.Run the MyQueueSender driver to place an XML message on the configured JMS Queue. This can be achieved by the following steps:
In Studio, right-click the Bar template and select Instances.
When the workflow instance appears, double-click the instance and check out the contents on the ebXML message received (see Figure 36.19).
The B2Bi plug-in extends the Studio to allow specifying nodes, actions, and events for collaborative workflows to be used among trading partners.
The plug-in also integrates the BPM process engine with the conversation flow.
The B2Bi Console is a browser-based component that enables configuration (including creation) and monitoring of various functional aspects of B2Bi functionality. The B2Bi Console, shown in Figure 36.20, allows you to create, configure, and monitor business conversations, trading partners, collaboration agreements, and so on.
Business Connect is a lightweight client that can be deployed by an enterprise (for example, a trading partner), enabling it to be “integration ready” with a minimal software footprint. Business Connect involves a noninvasive installation with a low cost of management and maintenance. Business Connect also includes an embedded Web server and a database server. It can interoperate with partners’ existing applications. Document exchange is supported with existing environments, including JMS, MQSeries, Java APIs, FTP, and so on.
WebLogic Integration provides powerful functional components for both data and business-to-business integration. Data integration provides functionality to translate binary data from legacy systems into XML and transform XML data from one grammar (encoding) to another. B2Bi functionality facilitates trading partners to collaborate in real-time by providing a standards-based framework for managing interactions based on standard business protocols.