Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project's goal is to simplify the development of big data applications providing a lightweight runtime environment that is easily configured and assembled via a simple DSL.
Spring XD features include:
- High throughput distributed data ingestion from a variety of input sources into big data store such as HDFS or Splunk
- Real-time analytics at ingestion time, e.g. gathering metrics and counting values.
- Workflow management via batch jobs. The jobs combine interactions with standard enterprise systems (e.g. RDBMS) as well as Hadoop operations (e.g. MapReduce, HDFS, Pig, Hive or Cascading).
- High throughput data export, e.g. from HDFS to a RDBMS or NoSQL database.
The overall Spring IO foundation layer is a cohesive set of APIs and embeddable runtime components that enable the ability to build applications addressing a wide range of enterprise requirements. For any given application developers will only need to include those parts of the foundation that an application actually requires. The foundation makes it easy to identify components that work together; it does not force the use of components not otherwise needed.
Spring IO brings together the core Spring APIs into a cohesive and versioned foundational platform for modern applications. On top of this foundation it also provides domain-specific runtime environments (DSRs) optimized for selected application types. Spring IO is comprised of the Spring IO Foundation and Spring IO Execution layers.
Spring IO is 100% open source, lean, and modular.
All of the major workload types are represented: enterprise integration, batch processing, big data processing, web applications (including WebSocket support), and the creation of server-side APIs in support of single-page apps, mobile clients, and services.
The Spring IO Execution layer provides domain-specific runtimes (DSRs) for applications built on the IO Foundation modules. A DSR may run standalone without requiring deployment to an external container. The Spring IO DSRs include Spring XD, Spring Boot, and Grails.
Spring XD provides a powerful runtime and DSL for describing big data ingestion and analytics, export, and Hadoop workflow management. In many cases, descriptions of streams, taps and jobs can be directly pushed to the XD DSR without requiring any code to be written at all. The set of XD modules (components that can be used in DSL expressions) is easily extended using building blocks based on Spring Integration and Spring Batch.
Spring Boot reduces the effort needed to create production-ready, DevOps-friendly, XML-free Spring applications. It simplifies bootstrapping of Spring projects with minimal code, implements an extensible set of operational features such as automated health checking and metrics endpoints, and supports embedded containers enabling the creation of self-contained executables.
Grails provides a productive and stream-lined full-stack web framework by combining the power of the Spring IO Foundation components with a set of comprehensive Groovy-based DSLs.