6.5. Guidelines for Integration
Now that we've examined integration design approaches and techniques, let's look at some guidelines for integrating enterprise applications with EIS systems.
We examine guidelines for integrating legacy systems, particularly home-grown systems, and guidelines for using command beans. Although not as common, we mention guidelines for achieving integration by using metadata, registries, and versioning and evolution.
6.5.1. Integrating Custom Legacy Systems
Most enterprises have home-grown or custom EISs, also referred to as one-off systems, and these systems need to be integrated. As we have already seen, connectors are the best solution for integrating EISs in a J2EE environment. However, because these are custom systems, there are no off-the-shelf or vendor-provided connectors available for them as there are for well-known EISs. As a result, you may have to write your own connector for a home-grown EIS. The recommended way to write a connector is to use a connector builder tool. A tool such as this not only generates a connector, but (with the help of other J2EE application server services) it may also create a Web services layer.
Generally, there are two scenarios involving the integration of custom systems. In one case, two EIS systems need to interact with each other. The connector approach works well with this type of scenario: You merely write Java classes implementing the business logic for one EIS and use a connector to communicate to the other EIS system. In the second scenario, you intend to have several applications use a custom EIS. For this case, it is better to plug the custom EIS into the application server and expose the EIS's reusable business logic through the application server. You can expose the logic either by using enterprise beans with JMS or by using a Web services approach. If the situation warrants it, you can use both enterprise beans with JMS and a Web services approach together.
6.5.2. Using Screen Scraping for Integration
There are times when you may want to integrate an EIS that has no programming interface. Often, this happens with legacy mainframe applications. In these cases, you can resort to screen scraping to create a programming interface. With screen scraping, you write an adapter layer that acts as an end user entering data into the mainframe application, and this adapter layer serves as the programming interface. You then write a connector that uses this programming interface to accomplish its integration actions.
When resorting to screen scraping, be sure to keep in mind the limitations of the legacy system. Although you can integrate a new application to a legacy system, the original system has limitations that may make it unable to handle the new functionality. For example, suppose you used screen scraping to integrate a front-end application with a mainframe application designed to be interactive. The front-end application, since it has no human constraints, can suddenly pump in a high volume of requests to the mainframe application, which may not have been designed to handle such a load. It is hard to detect a problem such as this until runtime, when the system is suddenly overwhelmed with a high volume of requests. By this point in the development cycle, it may be quite expensive to fix the application.
6.5.3. Metadata
Metadata is information that describes the characteristics of a data set. For an EIS system, metadata describes the content, quality, and condition (among other characteristics) of the EIS's data.
EISs provide a range of access to this metadata. Some EISs provide programmatic access to their metadata, while others provide the metadata information in text form only, as a form of documentation. Sometimes, the metadata information is available through database system support tools. For example, some databases allow queries to learn its table names and schemas. Similarly, an EIS may give you access to its various quality of service parameters.
Metadata is often relevant to integration. Metadata allows you to use tools to discover properties of enterprise systems, and from this discovery to create appropriate, easy-to-use façades to the systems. By having access to metadata, tools can generate more meaningful classes.
Web services are designed to support metadata. Web services rely on WSDL files, which essentially provide metadata describing the services, the operations offered by the services, parameters for these operations, and so forth. Since these WSDL files are XML documents, they are accessible to tools and other programs. (See “Web Services Description Language” on page 36.)
Using an integration layer helps make metadata more explicit. When metadata is implicitly associated with EIS data, tools may have a difficult time discovering the metadata. For example, a database table column may represent distances from a certain point, and these distances are implicitly measured in miles. A tool that accesses the data from this table column may not be able to determine whether the distance is measured in miles or kilometers. By creating an integration layer, you can make this information explicit. You can name the methods that access the data in such a way as to indicate the associated metadata. For example, rather than have a getDistance method, you can call the method getDistanceInMiles. For Web services, the standard document formats that describe the service are designed to make the metadata more explicit.
Another way to make metadata explicit and accessible, especially for medium to large enterprises, is to store the metadata in a central location, such as a traditional LDAP directory. (LDAP, which stands for Lightweight Directory Access Protocol, is an Internet protocol that programs can use to look up information from a server.) You can make your EIS's metadata available to others in several ways. You can enable metadata support in your enterprise systems. The WSDL file that accompanies a Web service already defines metadata for the service. You may also provide specific methods that retrieve metadata related to quality of services, parameter constraints, and so forth. Publishing a schema for a document that a service accepts is another way to provide metadata.
When deciding what information to store in a directory, try to store data that is useful to more than one application. Storing metadata in a directory also helps if you need to analyze the impact of any changes to the application.
Also, consider implementing directory-based data sharing, since this enables applications to operate collectively. Such data sharing reduces management costs and improves an enterprise's overall responsiveness to business change. If you use this approach, be sure to set up authorization policies to control access to the data.
It is also important when using a directory service to locate authoritative or reliable sources of the data. A major reason for not using a directory service is outdated or suspect data. Users quickly learn that they cannot rely on the service and hence stop using it. Keep in mind, however, that a single application or database is rarely authoritative for all required data, and this may cause a data integration problem of its own.
6.5.4. Using Registries for Integration
You may want to consider using registries to integrate an application with an existing EISs, especially when you want a loose binding between the application and the systems with which it is integrated. For example, in the Web services approach, you can store the URL and the WSDL for the Web service in a UDDI registry. However, using a registry comes with the additional overhead of running and maintaining a registry server and the added programming complexity to have your application use the registry. The registry server may also be a single point of failure for your system.
Generally, it is not worthwhile to use registries for a small enterprise whose applications integrate with just a few EISs. Registries make more sense for medium and large integration architectures. You want to maximize the use of the registry among as many applications as possible.
6.5.5. Versioning and Evolution
Applications integrated with EISs change over time, as do the EISs. As these systems and applications evolve over their lifetimes, new versions of them emerge. Integration of the applications and EISs must be able to handle this evolution and versioning process.
Many enterprise applications change because of changes to the underlying business requirements. EISs change for similar reasons, too, but they may also change because of hardware and software upgrades and bug fixes. New functionality may be added to applications and existing functionality retired. Sometimes policies change and, as a result, applications must access EISs in a different manner.
Generally, it is easier to evolve the various components—the enterprise applications and EISs—than it is to evolve the integration layer, since it is the point of stability. Let's first look at some strategies to evolve EISs, and then discuss how to evolve the different types of integration layers.
Sometimes as it evolves, the changes to an EIS are internal only. The external functionality—the functionality visible to other applications—remains the same. When this is the case, you should strive to keep unchanged the original external interface specified in the integration layer. To do this, make the sort of changes to the internal implementation of the EIS that do not impact the external interface; that is, the internal implementation should be changed so that it adapts to the original external interface.
It is different when the integration layer evolves. One way to handle this is to use a transformation layer, which is an additional layer added to enable older applications to continue to work. A transformation layer accepts messages in the older style used by these applications and transforms them to the current format. Often, because of their looser coupling, it is easier to evolve integration layers that are Web services than other types of integration layers.
When you control both ends of the integration point (the Web services endpoint and the client), it is often easier to upgrade them both at the same time. When just the internal implementation of the Web service changes—and these changes have no affect on the external interface—you merely plug in the new implementation.
Internal implementation changes to the Web service that cause changes to the external interface can be relatively easier to handle if these changes enhance the service but do not modify the original service contract. In these cases, you can provide two sets of WSDL files, one describing the original service and the other describing the new, enhanced service.
However, more complex scenarios exist, and you cannot expect clients to immediately migrate to a new service implementation. Some clients may never migrate to the new service. For more complex scenarios, a good strategy is to publish a separate Web service endpoint that provides the new service version. Clients can migrate to the new service when convenient for them. You can keep the old service interface but re-implement the existing endpoint to use the new implementation of the service. You may even be able to publish the new endpoint under the same URL without breaking the existing clients. For example, if you are just adding some additional ports to your WSDL, you should be able to update the old WSDL with the new WSDL description and then existing clients should continue to work with no problems.
6.5.6. Writing Applications for Integration
One area in which you have quite a bit of flexibility involves how you code your new EAI application. Since the application is new, you have the maximum amount of choice for implementing how the application will access the various resources that it requires. The J2EE platform, which is moving in this direction, provides some facilities to help with this task.
The Java Business Integration (JBI) Systems Programming Interface, based on JSR 208, extends the J2EE platform with a pluggable integration infrastructure using WSDL-based message exchange. JBI is of most interest to integration-independent software vendors (ISVs) rather than enterprise developers. JBI enables ISVs to write integration modules that support business protocols, such as Business Process Execution Language (BPEL), and plug them into a J2EE application server using JBI mechanisms. JBI also provides these integration modules with useful services such as threading, context management, security, lifecycle management, connection pooling, timers, and so forth. As a developer, you deal with the Web services for the various JBI-based integration services provided by these ISVs.