Big data analytics (BDA) is used to build competitive advantage, drive innovations, and enhance revenue. This capability offers a cost-effective cloud-based solution for data analytics. As big data grows in importance, organizations are striving to derive meaningful insight from this data. This is crucial to enabling an ability to respond to real business needs in a timely manner.

The following diagram illustrates a simplified enterprise cloud architecture for a BDA environment and contains three network zones: public, cloud service provider, and enterprise:

This architecture is similar to that of a data lake deployment. Both structured and non-structured data are used as sources. Data is then staged and transformed by data integration and stream computing engines, after which it is stored in various repositories. The data may also be augmented, transformed, correlated, and summarized until it is finally made available to consumers through APIs. Cognitive computing technologies such as machine learning and natural language processing can also be used to automate ingestion, integration, discovery, and exploration.

This architectural approach is applicable to the entire analytics life cycle and can be applied as a data lake solution for a DevOps environment. The latter is achieved by adding metadata and semantic definitions to the enterprise data repository descriptions that are stored in a service catalog. These catalog entries are then augmented with governance classifications, rules, and policies that automate data management as it flows in, out, and through the data lake. The following sections give a summary of each of the required components.