Spring Cloud Data Flow helps in establishing message flows between different kinds of microservices created using Spring Cloud Stream.
Built on top of popular open source projects, Spring XD simplifies the creation of data pipelines and workflows--especially for Big Data use cases. However, Spring XD has challenges adapting to newer requirements (canary deployments and distributed tracing, for example) related to data pipelines. Spring XD architecture is based on a run-time dependent on a number of peripherals. This makes sizing the cluster a challenging exercise. Spring XD is now resigned as Spring Cloud Data Flow. The architecture of Spring Cloud Data Flow is based on composable microservice applications.
Important features in Spring Cloud Data Flow are as follows:
- Configuring a stream, that is, how data or events flow from one application to another. Stream DSL is used to define the flow between applications.
- Establishing a connection between the applications and the message broker.
- Providing analytics around applications and streams.
- Deploying applications defined in streams to the target runtime.
- Support for multiple target runtimes. Almost every popular cloud platform is supported.
- Scaling up applications on the Cloud.
- Creating and invoking tasks.
Sometimes, the terminology can get a little confusing. A stream is an alternate terminology for a flow. It's important to remember that Spring Cloud Stream actually does not define the entire stream. It only helps in creating one of the microservices involved in the entire stream. As we will see in the next sections, streams are actually defined using Stream DSL in Spring Cloud Data Flow.