Chapter 1. The Agile Enterprise

Agility is an essential quality of the enterprise of the future. An agile enterprise rapidly adapts to changing business challenges and opportunities. It continuously improves to optimize cost, quality, and speed of delivery. It enables top management to quickly implement new strategies and control key business parameters to gain competitive advantage. Agility resolves some common business challenges faced by many enterprises. But the agile enterprise does not fit current business models. It requires a new business paradigm—a new way of thinking about the business and new ways of planning, organizing, operating, and controlling the business.

SOA, BPM, and MBM are important aspects of this new agile enterprise business paradigm that is enabled by the supporting technologies. The change in thinking applies existing concepts and develops new concepts and relationships developed in considerable detail in the remainder of this book. This chapter introduces basic concepts.

These concepts and relationships are applicable across many industries. Manufacturing provides a rich diversity of business functions and challenges, and it touches on most other industries. In financial services, much of the ability to develop and deliver new products depends on information technology. Telecommunications and financial services typically have great opportunities to exploit SOA, since these companies tend to experience many unresolved mergers and acquisitions. The telecommunications business is just starting to undergo major transformation with wireless technology and convergence with cable, the Internet, and entertainment.

The telecommunication industry has also developed eTOM, the best practices process framework that provides a good starting point for definition of services, as described in Chapter 2 and elsewhere. The nature of telecommunications has caused that industry to be more receptive to development of standards and to be less secretive about its practices.

Regardless of the industry, top management must understand the necessity of agility, assess the current state of the enterprise, and commit to a transformation that may take a number of years. This chapter provides a foundation for later chapters by identifying major business opportunities to be realized by the agile enterprise, positioning its emergence in the evolution of information technology, and outlining the new way of thinking that is the basis of the agile enterprise. Later in this chapter we introduce the SOA Maturity Model, which provides a basis for assessment and planning phases of improvement leading to enterprise agility. Finally, several critical success factors (CSFs) are suggested to drive the transformation.

Readers are probably familiar with the information technology notions of service-oriented architecture (SOA), business process management (BPM), and model-driven architecture (MDA) because so many IT organizations are investigating these technologies, if they haven't already begun adopting them. SOA technology has enabled rapid and flexible integration of systems across organizational boundaries. BPM technology is improving flexibility and optimization of business processes. MDA technology is an enabling technology for MBM. MDA introduced standards for generating applications from models and more recently supports business modeling languages.

The current awareness of and experience with these technologies is a good thing, for two reasons. First, it means that most readers are familiar with the basic concepts as well as the reasons behind applying them (reuse, consistency, economies of scale) for IT cost reductions and systems flexibility.

The second reason is that in applying these technologies, IT organizations are beginning to understand that realization of the full business value of these technologies requires changes in the operation of the business. The traditional delivery of information technology is bottom-up, opportunistically introducing automation and integration but leaving the design of the business fundamentally the same. The new economies of scale and flexibility are not just in the use of shared code and component software architecture but in consolidation of business functions and an adaptive business architecture. Applying SOA, BPM, and model-based management (MBM) to create the agile enterprise requires a transition to a “top-down,” business-driven approach that puts bottom-up automation, integration, and optimization in a proper business context.

Thus readers can benefit from this book no matter where an enterprise is on the adoption curve for SOA, BPM, and MDA technologies. Whether an organization is at the investigation, design, or implementation stage, it is never too late or too early to use these IT approaches as a springboard to creating an agile enterprise. And though many enterprises will achieve significant benefits from implementing these IT technologies, the most successful enterprises will be those that exploit these technologies to achieve enterprise agility.

When Agility Pays off

The following discussion outlines several major challenges faced by many enterprises today. An agile enterprise is prepared to face these challenges, mitigating the risks and realizing the opportunities.

Consolidations

A major source of business benefit in early adoption of SOA, BPM, and MBM is through consolidation of redundant business operations. Opportunities for consolidation are particularly prevalent as a result of mergers and acquisitions. Typically, the combined enterprise organization reflects aggregation without consolidation. This is common in financial services companies, telecommunications companies, and information technology companies. But the synergy and economies of scale that might have been envisioned are typically not achieved, because each organization continues to operate in its own silo, each with its own computer applications. Large corporations with decentralized divisions or product-line organizations often have similar opportunities for consolidations.

With considerable effort, some consolidation of operations may occur over a period of years. But because mergers and acquisitions occur frequently, especially in the industries noted, it is difficult for operational consolidation efforts to keep up.

In contrast to the norm, an agile enterprise is able to define a plan for consolidation and consolidate key redundant operations very quickly—perhaps even before the merger or acquisition is finalized. The agile enterprise achieves this at a fraction of the time, cost, and risk experienced by a conventional enterprise. If each of the original companies has an agile architecture, this consolidation can be faster.

Consolidation is the primary source of benefits in the early stages of SOA adoption. Table 1.1 outlines benefits of consolidation that were captured by the SOA Consortium. These examples highlight actual projects in various industries.

Table 1.1. Examples of SOA Benefits Through Consolidation by Industry

Industry	Realized Benefits
Automobile	Improved customer satisfaction
Reduced duplication of customer data and near-real-time access to vehicle information
Increased agility through a governance focus
Easier integration with partners
Energy	Flexibility and speed in changing business processes
Business optimization and risk mitigation: accurate real-time commercial, financial, and profitability data across the value chain
System reliability: simplification of interfaces by duplicate master data reduction
Pharmaceutical	Improved visibility into product line
Increased agility in taking pharmaceutical products to market
Cost savings and reduced headcount
Better use of core architecture, providing improved data integration, management, and reusability
Achieving 99.999% uptime on a stable platform
Telecommunications	Elimination of network outages
Stronger focus on strategic initiatives while reducing cost of IT operations to 30% of previous level
More transparency by masking systems complexity from users $80 million in value over two years from improved efficiency, responsiveness, and adaptability of the organization
67% reduction in mobile phone provisioning costs
50% reduction in cost of third-party development bids, and faster development times (hours vs. weeks) due to SOA environment and automated tools
Faster time-to-market for new services
Seamless migration to a convergent system of prepaid and post-paid customers
Lower maintenance costs
Improved scalability
Transportation	Flexibility and speed in providing new services to customers
Ability to grow higher-margin businesses in the United States and overseas
Reduced cost of supporting infrastructure for internal/external customers
Easier integration of acquisitions though a common core set of services
Rapid transformation and reuse of processes and services
Elimination of errors and shortening of response cycle through automated processes
Significantly scaled-up usage of self services and end-to-end process integration
Entertainment	Consolidation of multiple content rights systems into one 50% decrease in time needed for year-end accounting closure

In the short term, as an enterprise is moving toward agility, consolidation of redundant capabilities is a major source of value, even when the enterprise is still in the early stages of transformation. These consolidations will often have an IT focus, but they necessarily involve the consolidation of the associated organizations. This will demonstrate the business value of shared business capabilities and is representative of the current level of transformation of most early adopters. Other benefits of SOA, discussed in the following sections, are not as apparent in the early stages.

New Product or Line of Business

Top management may recognize an opportunity to introduce a new product or enter a new line of business in an emerging marketplace in a way that builds on some of the key strengths of the current enterprise. Though some weaknesses will need to be addressed, rapid entry into a new market will be critical to long-term success.

A traditional enterprise might address this opportunity by forming a separate division or acquiring an existing company to avoid the burden and risks of adapting existing operations to the new market, because existing processes and computer applications are designed to optimize those current lines of business. However, at the same time, smart management understands that a new business silo cannot utilize the strengths and potential economies of scale of the parent enterprise.

The agile enterprise is able to engage existing capabilities of the enterprise in the new line of business without penalizing the existing business. Top management is able to quickly assess the impact of the new business, determine realistic operating costs and competitive pricing, assess the required investment, and implement the new product-line capability.

The benefits of agility in introducing a new product line or business have been realized in a number of industries, including telecommunications, pharmaceuticals, and transportation. The benefits include (1) increased visibility and control into the product line, (2) the ability to utilize a core architecture to improve data integration and consistency of implementations, (3) significant improvements in development schedules and time to market, and (4) higher customer satisfaction, in part due to reduced cost of using services.

Outsourcing

Much of the cost of doing business goes into necessary operating activities that are not part the enterprise's core business and do not provide competitive value. Business operations such as finance and accounting, human resource management, and information technology require special skills and are increasingly complex, particularly for multinational enterprises. At the same time, these activities require considerable management attention and are challenged to achieve industry best practices for regulatory compliance, efficiency, and effectiveness.

Large enterprises have adopted outsourcing as a long-term strategy to mitigate these problems. IT outsourcing has been adopted in all industries. Outsourcing of financial and human resource management services is gaining in popularity. The agility benefits for outsourcing include (1) scalability—the ability to quickly accommodate increased or reduced workload, (2) expertise—outsourcing providers can maintain skilled people to deal with change such as regulatory requirements, and (3) internationalization—a outsourcing provider should be prepared to support the client in expansion into new countries. It should be noted that small enterprises and startups in all industries can benefit immediately from agile outsourcing thus reducing the barriers to entry of new competitors.

Outsourcing offers the opportunity to exploit the expertise and economies of scale of a service provider while reducing the management burden associated with these operations. However, these supporting services are often intertwined throughout the enterprise, and the division of responsibilities may be inconsistent across the enterprise. The disruptive effect of a transition to an outsource service provider could have a major impact on the rest of the business.

The agile enterprise is able to quickly identify the business activities to be outsourced and their relationships to other business activities. The business units are components of the enterprise, just as an engine is a component of an automobile. A more powerful engine might require some changes in other automobile components, but the relationships to the controls and other components should be relatively easy to identify, evaluate, and resolve.

Thus outsourcing is another source of substantial business value that can be realized in the early stages of enterprise transformation, as long as the integration is compatible with a strategic information technology infrastructure.

Government Regulation

Government regulation is an increasing concern. Managers are being held responsible for the integrity of their operations and protection of stockholder interests. Multinational enterprises must comply with business regulations of countries in which they operate as well as regulations for products or services in countries in which they sell. Not only are regulations constantly changing, but changes to the business organization itself can create risks of violations. Regulatory compliance affects all industries.

Implementation of compliance is a challenge in conventional organizations because the affected processes may be undocumented and may be performed in multiple organizations in different ways.

The agile enterprise is able to quickly and reliably assess the implications of regulations to the business and plan appropriate changes and controls to ensure compliance. The consistent business architecture, along with consolidation of sharable business capabilities, clarifies responsibility and accountability for compliance. Formally defined business processes and business process automation support the implementation and enforcement of regulations.

An important aspect of regulatory compliance is reliable recordkeeping. Formal definition and automation of business processes support the capture of appropriate records. Electronic identity and signatures ensure proper authorization and accountability for record content.

Outsourcing regulated activities such as accounting, human resource management, and IT reduces an enterprise's burden and provides greater assurance that appropriate expertise is applied to implementation of regulations and related changes.

Governance

To optimize enterprise operation and ensure appropriate accountability, control, and agility, enterprise design requires a disciplined approach and consistent architecture.

The conventional enterprise reflects adaptations from enterprise design that, in many cases, predates the use of computers. Responsibility for continued design has been delegated to large departments or lines of business that focus on optimization within their local spheres of influence. Large departments or product lines tend to be geographically isolated or located in separate buildings so that their internal capabilities are easily coordinated. In many cases, capabilities are developed rather than shared because it is easier to develop and adapt a capability if you own it yourself.

Electronic technology has substantially reduced the barriers to coordinating and sharing capabilities, but many opportunities for improvement, particularly from an enterprise perspective, have not been realized. Organizations will resist shared capabilities because they represent a loss of control. Furthermore, this evolved organizational design has made the enterprise increasingly complex, making it difficult to maintain accountability and control and difficult to adapt to changing business needs.

For example, a large information technology company entered the IT services business and incurred substantial losses. The board of directors was not aware of the extent of the losses because the losses were obscured by profits from its successful hardware business. In contrast, an agile enterprise has a clear picture of product costs so there is recognition of successes and accountability for failures. The agile enterprise is also better equipped to assess the challenges and risks of a new product or line of business because applicable existing capabilities and new capability requirements can be defined as elements of a rigorous product value chain.

The agile enterprise has a consistent architecture and is composed of service units with well-defined interfaces and performance objectives. This provides a consistent basis for evaluation of performance and accountability. In addition, the performance of service units can be evaluated in the context of contributions of value to customer products and successful operation of the enterprise. Consequently, top management has a clear view of the operation of the enterprise and its strengths and weaknesses, and the board of directors can better assess whether the enterprise is doing the right thing and doing it well.

Technology Modernization

Technology modernization may encompass any technology upgrade or improvement to a business capability. Many enterprises are captive to information systems developed long ago, many of which have locked the enterprise into ways of doing business that were optimal at the time but have since become outdated. The design of the systems as well as the technology used to implement them may be obsolete and difficult to support or change.

Obsolete technology is a challenge in enterprises in every industry. The challenges are particularly pronounced in industries such as financial services and telecommunications, where there have been multiple mergers and acquisitions, with systems implemented in different technologies and tightly coupled to particular product lines or markets. The challenge becomes not only the upgrade of technology but consolidation of the business logic and processes and integration of consolidated solutions with the remaining legacy systems that support different lines of business.

Duplication of functionality also occurs where the legacy systems cannot easily be adapted to support new lines of business, so the legacy functions are duplicated in new systems. Replacement of legacy systems almost always requires major investments and entails substantial risk. But replacements of legacy systems without also providing enterprisewide shared services only leads to more inefficiency and inflexibility.

The agile enterprise makes business processes visible and adaptable and relies on more focused and finer-grained applications that can be individually upgraded or replaced without major upheavals. Here, application modernization tools can help in the transformation of legacy systems, making business processes more visible and supporting more finely grained shared services.

Agility in technology modernization, like governance, tends to provide more benefit to SOA-mature organizations because fewer business processes are embedded in applications and because applications (and other technology) will be more fine-grained and therefore replacement can be more limited in scope. This makes it possible to replace a capability's implementation with less impact to related services.

How We Got Here

It is useful to consider the evolution of the business use of information technology to understand how the current “hairball” of systems and communications developed over time and why the time for the agile enterprise has come. This mish-mash is the legacy that we must transform to realize the agile enterprise.

Task Automation

Early, widespread applications of the computer were for task automation. The computer could do monotonous, repetitive tasks faster, cheaper, and more reliably than people could. Computers were kept in controlled environments, and people brought the work to the computer and picked it up when it was done.

As more tasks were automated, they were bundled together into increasingly large applications. People interacted with the applications online, so the data stayed with the applications and was eventually stored in departmental databases. Some workflow management systems emerged to direct the flow of records between tasks performed by people. But most of the flow of work between the tasks was built into the systems, sometimes through the transfer of magnetic tapes and sometimes embedded in program code.

Large applications grew within departments to streamline their operations, and files were transferred between departments, initially on magnetic tapes and later through electronic transfer of files. The movement of files between applications was automated for efficiency and control. Within large applications, embedded business processes could move transactions between tasks as they occurred, but records were still batched for transfer to the applications of other organizations.

The transfer of files between applications extended outside the enterprise, to suppliers, large customers, health care insurers, and financial institutions. Industry standards were developed for electronic data interchange (EDI). File transfers were typically a daily occurrence—batches of records from the day's business activity. This movement of files between applications was generally point-to-point communications, as depicted in Figure 1.1a. For remote applications, the communications occurred over dedicated telephone lines.

Figure 1.1. Transition from Ad Hoc Integration to EAI Middleware Integration.

Enterprise Application Integration

Enterprise application integration (EAI) middleware emerged in the marketplace. It brought the hub-and-spoke communication model depicted in Figure 1.1b. Within an enterprise, the middleware could route messages from many sources to many destinations, reducing the number of communication links and improving control. In addition, there was no longer the need to send records in batches, but individual records could be sent as messages as they became available.

EAI middleware enabled a transition from batch-oriented enterprise integration to transaction-driven integration. The EAI middleware provides a buffer so that a message can be sent when a receiver is not yet ready to receive. It can also provide a buffer between legacy batch processing systems and those systems that process and send transactions as they occur. EAI middleware products provide adapter software to integrate systems implemented with diverse technologies and message transformation services, to make the data structures compatible between applications. Transaction-driven systems accelerate the delivery of results; for example, a customer order for stock items might be processed and the order shipped the same day.

Of course, the hub-and-spoke configuration relies on the use of shared middleware. Unfortunately, this is a barrier to integration between enterprises and sometimes within a large enterprise, particularly in the absence of interoperability standards for message exchange.

The Internet

As EAI was gaining widespread adoption, the Internet and the World Wide Web were gaining momentum. The Internet opened the door to many-to-many communications in a different way: The public Internet was the global hub through which messages could be directed from any Internet subscriber to any other Internet subscriber. Dedicated telephone lines were no longer needed between business partners.

There is no industry standard for message exchange using EAI middleware, so a standard format was required for communicating between diverse systems over the Internet. Web pages were already being communicated between diverse systems, so this technology was adapted to communication of messages between business systems.

Hyper Text Transport Protocol (HTTP) from the Internet Engineering Task Force (IETF) and the World Wide Web Consortium (W3C) became the accepted messaging protocol, and HyperText Markup Language (HTML) from W3C became a basis for exchange of content; it was already allowed to pass through corporate firewalls for Web access. Since the messages were not intended for graphical display, HTML per se was not appropriate for application integration, but eXtensible Markup Language (XML), also from W3C, shares the underlying technology of HTML that enables interpretation by diverse computer systems, but it also provides greater flexibility for content specification and transformation. XML is discussed further in Chapter 5.

The Internet became the medium of exchange for business-to-business communications. IT industry leaders recognized a potential for ad hoc relationships between businesses to be established automatically, at a moment's notice, if only there were industry standards by which these relationships could be specified and discovered.

Web Services and SOA

The concept of “Web services” emerged. Figure 1.2 illustrates the vision. The arrows depict request-response relationships. In concept, an enterprise posts a service offering on a public registry. Another enterprise in need of a service queries the registry to obtain information on available services. The registry includes information about the service and the protocol for using the service. The service user then sends a message to the service provider, initiating the exchange. All this is expected to be performed automatically by applications of the participating enterprises. Within the enterprises, the exchanges are mediated by automated business processes.

Figure 1.2. Web Services Vision.

Standards for Web services have been developed, but the ad hoc, automated selection of services has not caught on. Business leaders are not ready to trust computer systems to establish and manage business relationships, not least because the current abilities to express the actual semantics of a given service offering leaves a lot to be desired. However, much of the technology has been adopted, and relationships established by humans can quickly be automated for exchange of business transactions over the Internet.

The concept of accessing services over the Internet and the development of standards for communications with services led to a new kind of service-oriented architecture (SOA) in which Internet-based technologies support an integration of systems offering services and using services. Not only can the technology be used over the public Internet, it can be used within enterprises to integrate systems. This greatly expanded the market for a new breed of middleware to perform Internet-based communications and drive interactions with automated business processes.

Within the enterprise, SOA has been viewed as a way to implement shared application components. Functionality used in different areas of the business could be implemented as shared services and invoked by other applications. Web services technology was promoted as an alternative to EAI middleware. Rather than route all messages through a central hub, Web services technology would enable direct communications between applications over the enterprise intranet (an internal network using Internet technology). The concept of the enterprise service bus (ESB) emerged as middleware that enabled applications to be connected over the intranet using Web services technology. Essentially, an ESB is decentralized EAI middleware with standards-based communications.

However, the major impact of SOA will be realized as a business architecture rather than an IT architecture. For the business community, SOA is an approach to the design of an enterprise in which distinct business capabilities are offered through well-defined mechanisms and media of exchange so that the capabilities can be used in multiple business endeavors now and in the future. This enables the capabilities to be managed for consistency and economies of scale, allowing the enterprise to more easily adapt to new endeavors by making these capabilities shareable.

The idea of SOA as a business architecture is beginning to emerge in the industry. Unfortunately, though the potential benefits are great, the challenges are also great. The principal challenge is to change the way both business people and IT people think about the way the enterprise is organized and managed and the way it is supported by IT. We return to these concepts in the discussion of SOA infrastructure in the next chapter.

A New Way of Thinking

Intense competition, a changing world, complexity, and increased risks demand a new approach to enterprise management to optimize enterprise performance and agility. SOA, BPM, and MBM, with support from related information technology, enable and require a management paradigm shift—a change in the way of thinking about the operation and management of the enterprise.

SOA brings a fundamental change to the structure of the enterprise. Traditionally, an enterprise operates as a number of distinct departments or divisions, each with its own systems and business operations in its own silo, as depicted in Figure 1.3. In the diagram, A and B might be different divisions or product-line organizations. Each division has its own specialized business units, contributing capabilities to the divisional efforts. Some of these are duplicated between the divisions as indicated by the boxed letters, which represent business capabilities. Each division has its own applications used by people within the division. Interactions between divisions and with outside customers and suppliers are through well-defined channels. Access to the internal capabilities is restricted by locked doors and passwords.

Figure 1.3. Conventional Business Unit Silos.

Initially, these silos were connected through transfers of files; later, EAI improved the speed and flexibility of communications. But those communications, for the most part, are in controlled environments between known and trusted systems. Typically, the transfer of business transaction data from one department to another also transfers responsibility and control for the transaction. The sending system is trusted to send valid records and the receiving system accepts responsibility for subsequent action. The sources and destinations of these data transfers are well known and stable. Changes to business processes are restricted by the flow between systems and the hardcoded processes that integrate capabilities within the systems.

SOA (with the help of SOA technology) opens up these silos and makes capabilities within them available for use as sharable services. Similar capabilities can be consolidated for economies of scale and to achieve consistency across the enterprise.

As a result, a SOA has interorganizational interactions and sharing of capabilities at a lower level of granularity, as depicted in Figure 1.4, that represents a transformation from Figure 1.3. Each of these boxes represents an organizational service unit providing and/or using a service. The arrows represent request-response relationships.

Figure 1.4. SOA Network of Services.

Each of these service units can contribute their capabilities to address similar needs in different contexts. The divisions or product lines, A and B, still exist from a product management and marketing perspective, but they share common capabilities.

A service unit (further defined in a moment) participates as a service provider to support a requester's objective. The same service provider may participate as a requester to incorporate the capabilities of other services. Consequently, a request for one service may propagate to many other services. The smaller granularity of service units enables the capabilities to be more stable and usable in different contexts, both as the enterprise currently operates and in future business endeavors. The service units can be used selectively, instead of the larger, more complex “department” being adapted for the specific requirements of each new undertaking.

This sharing of capabilities across the enterprise has been enabled by information technology. Information technology has become an integral part of the fabric of doing business. It's no longer just about automation of tasks and faster communications. Information technology has changed the business equilibrium.

Traditional organization structures, work environments, access to information, international trade, and customer expectations have changed. They are still changing, and technology has accelerated the pace of change. Optimization of enterprise operations not only involves more variables, but the enterprise must continuously change to adapt to new challenges and opportunities. Agility has become a competitive necessity, and the bar is rising.

SOA, BPM, and MBM are important components of enterprise agility. Though SOA is being driven by information technology, it reflects a change in organizational responsibilities and relationships as a result of IT. BPM is primarily a business process management discipline, but it reflects technological support for monitoring, modeling, and automating business processes.

Though managers have financial models, MBM brings computer-based tools to manage complexity and to plan and validate rapid changes to business operations and relationships. Together these disciplines and the supporting technologies enable the more effective governance, leadership, planning, and decision making needed for the agile enterprise.

The new paradigm changes the design and management of the business. The following sections highlight some of the key concepts.

Service

A service is an application of a business capability to provide business value needed by a community of service users. A service is requested to fill a need. It is important to distinguish the service—the value provided—from the request for a service and the capability used to deliver the service, since we often call all of these services. Examples of potential services are when a customer receives a product, a machine is repaired, a product is designed, a package is shipped, an account is credited, benefits options are selected, or a purchase is completed. The corresponding requests are a customer order, a machine repair order, a product design request, a shipping order, an account credit transaction, a benefits selection request, and a purchase order. We define a service unit as providing the capability. In fact, every enterprise capability that contributes value in response to a defined business need can be designed to provide a service or a related set of services. Applying business capabilities through services allows the capabilities to be shared, enabling economies of scale and improved consistency, accountability, and control of operations.

Service Units

A service unit is a business unit responsible for management of a business capability to provide services—the processes, resources, facilities, intellectual property, computer applications, and operations that perform the service. The service unit is not expected to do everything itself but delegates some responsibilities to other service units for specialized capabilities and for support services, such as human resources, accounting, and information systems services, that support its capability. A service unit is a focal point of responsibility and control for the service delivery capability it offers. Offering capabilities as distinct building blocks that provide stable, sharable services enables the enterprise to adapt more quickly to changing business needs.

Service-Oriented Enterprises

An entire enterprise can be configured as a network of interworking service units to become a service-oriented enterprise. In a service-oriented enterprise, each service unit provides a well-defined mechanism by which services are requested, and it assesses costs for each service it performs. Economies of scale are achieved in the development, integration, and support of services, and the consistent architecture provides enterprise transparency, accountability, and control for more effective governance and agility. The consolidation and sharing of capabilities across the enterprise requires a top-down, enterprise perspective to properly select and scope service units, to avoid local suboptimization, and to overcome resistance to loss of ownership and control.

Agile Enterprise

An agile enterprise incorporates an SOA along with BPM techniques for process design and optimization and an effective governance structure. In addition, the agile enterprise uses MBM tools to manage complexity, support optimization of enterprise operations, and enable rapid reconfiguration of service units to respond to changing business threats and opportunities. Aspects of an agile enterprise are more fully developed throughout this book.

Process-Driven Services

A business process defines the orderly performance of activities that achieve a desired business result. Business processes drive the operation of each service unit and its use of other services. Each business process operates within the scope of an associated business unit (a service unit) that is responsible for the management and optimization of that business process. When another service is invoked, the action should be viewed as invoking a business process within the other service unit to access the capability of the target service unit for a particular purpose. Consequently, service units define a framework for the definition and integration of business processes.

Model-Based Management

MBM is the use of computer-based models of the enterprise to enable managers to understand, analyze, plan, and make decisions regarding the operation of the enterprise and to respond quickly and effectively to threats and opportunities. Models are linked to the business operations to reflect the current and evolving state of the enterprise, and models are used to plan, evaluate, and manage transformations of the enterprise.

Value Chain

A value chain is a dependency network of activities that contribute to the value delivered to a customer. A customer may be an internal business activity, so an enterprise has multiple value chains. A primary value chain defines the direct contributions to the value of a unit of production delivered to an end customer. In an agile enterprise, the value chain identifies the contributions of cost, quality, and timeliness of each of the participating service units. The value chain supports product costing and becomes the focus of analysis for pricing and competitive improvement.

There are different points of view on modeling the creation of value. Some of these have been given different names, such as value network and value stream. We believe the dependency network representation used in this book fits the agile enterprise architecture needs and captures the fundamental concepts to support different analytical viewpoints.

Disruptive Event

A disruptive event is an event that suggests the occurrence of an enterprise threat or opportunity. The agile enterprise recognizes, analyzes, and responds to disruptive events that occur in the enterprise ecosystem. Event resolution services must drive adaptive changes or bring the events to an appropriate level of management attention. Events that cannot be resolved by operational or tactical adjustments must be addressed by strategic planning activities that determine the need for more pervasive changes.

Governance

Governance is the set of responsibilities and practices exercised by top management and the board of directors to perform necessary enterprise design, oversight, and control to achieve the desired owner value. In the agile enterprise, governance is not simply the management of budgets, priorities, and sales objectives. Executive staff service units provide enterprise design, metrics, intelligence, modeling, and analysis capabilities as well as mechanisms for accountability and control. These services support effective guidance and leadership by the board of directors and top management. With effective governance, service units have clear responsibilities, visibility, and accountability for (1) management of their capabilities, (2) compliance with policies, laws, and regulations, and (3) delivery of results.

Information Technology Management

Effective utilization of information technology is the responsibility of top management as well as the managers of individual service units. The IT organization, and ultimately the CIO, have responsibility for making appropriate technology available to the service units and optimizing the utilization of IT resources. This availability is provided through supporting infrastructure, operation of data processing and communications services, and development and support of information technology applications that service units require.

The IT organization is itself an aggregation of service units and thereby leverages special skills and resources. The agile enterprise manages the application of technology for enterprise-level optimization of IT investments and economies of scale in the utilization of IT resources.

SOA Maturity Model

Transformation to the agile enterprise is a major, long-term undertaking. Many more books will be written about planning and managing transformation as well as the technology to support the transformation and implement the capabilities. Here we provide insight into the phases of transformation and the dimensions of change.

The SOA Maturity Model developed by EDS and Oracle is depicted in Figure 1.5. The model defines criteria to assess the degree to which an enterprise has realized the potential of SOA and associated disciplines. There are many paths to the future that are well beyond the scope of this book. The SOA Maturity Model provides guidance for planning the enterprise transformation and a basis for objective evaluation of progress.

Figure 1.5. SOA Maturity Model.

The SOA Maturity Model is similar in concept to the Capability Maturity Model Integration (CMMI), from the Software Engineering Institute and Carnegie Mellon University. The model provides criteria for assessment of (1) the readiness of an organization to accept and manage the associated discipline and (2) the degree to which the necessary organizational structure, disciplines, and supporting elements are in place.

Some reviews of this maturity model have failed to recognize the significance of the business perspective. It is the business perspective that sets this maturity model apart and provides a foundation for discussion of transformation in this book.

A maturity assessment based on this model identifies the maturity level of an enterprise and the issues that should be addressed to progress to the next level. Each level builds on the capabilities of the levels beneath it. Just as the construction of a building must start with the foundation, progression to SOA maturity and enterprise agility must progress up the levels of the maturity model.

Investments in enterprise capabilities are needed to support future advances. These investments increase some costs in the near term. At the same time, the transformation plan should achieve incremental improvements through projects that each realize business benefits along the way.

Each of these levels is assessed from two perspectives: business with five dimensions and technology with six dimensions. Each of these dimensions has criteria for assessment at each of the five levels.

The details of the intersections of levels and dimensions are not presented here, but we briefly consider each of the levels and each of the dimensions to provide some further insight on the phases of transformation.

Maturity Levels

An enterprise achieves a maturity level when it has substantially achieved the capabilities identified for that level. Though some of the capabilities of higher levels may also be achieved, the overall capability of the enterprise is still limited by those capabilities that have only reached a lower level of maturity. The level of return on investment is lower for transformations undertaken at lower levels of maturity, but the risks are higher if an undertaking is too ambitious for the current level of maturity. Each of the maturity levels is discussed briefly here:

1. Explored. An organization is aware of SOA and may be studying the potential impact or doing some proof-of-concept development.This is the current “status quo” level of most enterprises. Typically the SOA awareness is in the IT organization (in other words, it is awareness of SOA technology). If other organizations are aware of SOA, they most likely view it as another wave of technology. The business side of the enterprise is more likely to be focused on BPM and process improvement where automation of business processes is viewed as a technique to be considered, but the focus is on the operation of the business.A proof-of-concept development should be selected to demonstrate the business potential and organizational capability to consolidate and integrate a capability. In most cases, this will be driven by IT and will focus on consolidation of applications, but the business value and organizational implications of the consolidation should be highlighted. This includes economies of scale, consistency, and accountability as well as delegation to shared services (perceived as loss of control) and commitment to delivery of services in compliance with formal specifications.
2. Applied. Top management is committed to SOA, the organization has developed a basic capability to design and implement service units, and selected shared services are being used (bottom-up). The Maturity Model does not distinguish between a service as value delivered and a service as an organization responsible for the supporting capability; in this book we resolve that ambiguity by referring to the organization responsible for the supporting capability as a service unit, as discussed earlier.At this level SOA has become recognized as an important approach to improvement of operating costs, product quality, and agility of the enterprise. It may still be viewed as primarily an adoption of new technology, but there is a realization that it must be driven by top management to achieve strategic value and avoid suboptimal solutions. There is an understanding that SOA and BPM are complementary views of an enterprise architecture, and that service units are shared business capabilities managed by business organizations. There is an initial commitment to an SOA infrastructure and enterprise standards. Development of service units is essentially bottom-up, based on business value, and should be guided by an industry best-practices framework perspective.
3. Adopted. The organization has an SOA infrastructure in place and is committed to standards. There is a system of governance to plan and manage transformation of the organization and to manage the definition and implementation of service units (top-down).At this level, the transformation has shifted from being driven bottom-up to top-down. Definition of service units is driven by top-down analysis and design by a business architecture activity, and transformation is driven from an enterprise level. Priorities and funding for IT budgets and transformation initiatives are managed at an enterprise level. Service costs are captured, and a charge-back mechanism has been defined to support evaluation of the full cost of services. The enterprise is not yet fully service-oriented, but development of new applications is in a service-oriented context. Data exchange for established services is consistent with an enterprise logical data model.
4. Measured. Service units are monitored and measured for cost, timeliness, quality, and availability and refined for enterprise optimization; in other words, Level 4 capabilities are value chain driven. The contributions of services to the value chain can be reported and analyzed.The enterprise is sufficiently service oriented that the value chains can be evaluated as compositions of services. The cost, quality, and timeliness of a value chain are reported and can be traced to the individual service units that contribute value. The organization structure reflects alignment of goals, incentives, and economies of scale in the management of service unit resources. Service performance is monitored in real time, and performance is evaluated against formal service unit performance specifications. Disruptive events, both internal and external, are captured and directed to appropriate service units for resoltion.
5. Agile. The organization has a continuous change culture and business processes to adapt the enterprise in response to disruptive events. The enterprise senses disruptive events and, when required, responds to them by reconfiguring relationships between existing service units, with minimal need for capability enhancement or development of new services.The governance structure ensures that the enterprise is doing the right thing and doing it well. The enterprise accepts change as a way of life. Continuous strategic planning is responsive to change and drives strategic changes to the enterprise. There is rapid response to disruptive events through business processes based on comprehensive risk management and an understanding of the enterprise ecosystem. Service unit managers work to continuously improve their services based on needs of service users and enterprise objectives. Service units are sharable building blocks that enable rapid configuration, evaluation, and implementation of a product life-cycle model to address new business opportunities.

Business Dimensions

Each of the maturity levels is evaluated, from a business perspective, in five dimensions. The following points briefly describe the business dimensions:

• Processes. Business processes must first be documented and repeatable. They must then be aligned to service units and measured. The agile enterprise has business processes that determine the operation of the enterprise but also processes that drive change.
• Organization. The business organization evolves to an organization of service units that are later organized for effective management of capabilities and incentives for improvement.
• Governance. Governance evolves from delegation of optimization and change in business silos to enterprise-level planning, priority setting, accountability, and control, to achieve a consistent enterprise design and responsive, coordinated change.
• Portfolio. The portfolio of shared business capabilities goes from a functional organization chart to a well-defined collection of shared service units and, finally, a comprehensive model of a network of service units contributing to value chains.
• Finance. Funding of information systems evolves from departmental discretion to investment based on enterprise priorities. The cost of services is determined and supported by a billing mechanism for assessment of the full cost of each service rendered, including other services used.

Technology Dimensions

The six dimensions of the technology perspective are focused on particular concerns of the IT organization and the capabilities needed to provide information technology support to the rest of the business:

1. Infrastructure. Infrastructure moves from ad hoc, point-to-point integration of systems to a common messaging and integration infrastructure with single sign-on and role-based access control. Business process automation and, later, event notification and complex event processing are included in the infrastructure.
2. Architecture. The architecture evolves from support for integration of applications to design of technical solutions to support and integrate service units and, later, support for detection and resolution of disruptive events. Technical standards and product selections support economies of scale in IT development and operations.
3. Data. Data models evolve from project-driven data modeling to development and application of an enterprise logical data model that defines data exchanged between service units, the content of master data records, and the integration of data to support enterprise intelligence (which includes analysis of events and trends).
4. Governance. Technology governance evolves from project-based to program-based (multiple organizations and projects) to enterprise-based management of technology investments, standards, and product selection. Technology governance becomes an aspect of enterprise governance.
5. Organization. The IT organization evolves from a departmental/application focus to a capability focus with development of special skills to support the design and implementation of an agile enterprise.
6. Operations. The IT operations activities move from management of individual applications to monitoring and management of service unit dependencies and a virtualized computing environment. In the latter, computing devices are no longer dedicated to particular organizations or applications in order to enable operational economies of scale, dynamic performance optimization, and high reliability and security.

Critical Success Factors on the Journey to Agility

The transformation to agility is a journey up the levels of the maturity model. The roadmap for the journey differs for each enterprise because each enterprise faces different challenges. However, we can highlight some critical success factors (CSFs) to help top management drive the transformation in the right direction.

Governance for Enterprise Optimization and Control

Top management must ensure that investments, improvements, and economies of scale are considered from a strategic enterprise perspective. In particular, information technology must be managed to control proliferation of diverse technologies and to achieve economies of scale in technical resources. Departmental or line-of-business silos must give up control of duplicated capabilities to realize enterprise-level economies of scale and flexibility of shared services. Service units must be held accountable for compliance with service specifications, business rules, and security requirements.

Enterprise Models

To optimize enterprise operations and respond effectively to challenges and opportunities, top management must have models that provide information about the enterprise ecosystem, current operations, operating cost, quality and performance, and opportunities for improvements as well as new business. These models go well beyond the “executive dashboard,” to enable analysis of disruptive events, consideration of what-if scenarios, and exercise of operational controls. Value chain analysis must provide an understanding of the contributions to cost, quality, and performance for each current or planned product or service. Business activity monitoring should identify exceptions and trends in performance. Recognition of events and distribution of notices should keep top management aware of the changing ecosystem.

Technical Infrastructure

A shared technical infrastructure must be established and maintained for economies of scale, integration, flexibility, reliability, security, and support for robust enterprise intelligence. This infrastructure requires initial investment that cannot be justified for individual application development projects; it is intended for use by most or all applications. The technical infrastructure includes services such as messaging, security, naming, business rules repository, logging, and more.

Service-Based Management

Service-based management is fundamental to the paradigm shift. Managers must start to think in terms of providing services either directly to end customers or to other parts of the enterprise. This means formalizing capability offerings and the form of requests, responses, and related information exchanges. It means determining the costs of service units and the unit cost of using individual shared capabilities, including the costs incurred from other services used. It also means accountability for performance measures, security, compliance with policies and regulations, and responsibility for continuous improvement and adaptation to change.

Optimization of service-based management is enabled by an SOA, which is the focus of the next chapter.