Hazelcast Jet 0.5 API unveiled to increases dev productivity

Hazelcast has announced the 0.5 release of Hazelcast Jet - an application embeddable, distributed computing platform for fast processing of big data sets. New functionality in Hazelcast Jet 0.5 includes the Pipeline API for general purpose programming of batch and stream processing, and fault tolerance using snapshotting with the integrated Hazelcast IMDG. The overall focus of the latest release is to increase developer productivity via an extremely simple and intuitive API. Jet is a single library with no dependencies which is therefore easily embedded and deployed, removing the need for multiple systems. Typical application use cases include online trades, sensor updates in IoT architectures, real-time fraud detection, system log events, in-store e-commerce systems and social media platforms.

The Pipeline API is the primary programming interface of Hazelcast Jet for batch and stream processing, making it more appealing to a wider Java audience. This is a major enhancement to the Hazelcast Jet low-level Core API which uses directed acyclic graphs (DAG) to model data flow - allowing detailed DAG assembly of processing jobs. The new Pipeline API is easier to use and provides developers with tools to compose batch computations from building blocks such as filters, aggregators and joiners. The Java 8 Stream API is also available in Hazelcast Jet 0.5.

Greg Luck, CEO of Hazelcast, said: “Since its first release Jet has put the Fast in Fast Big Data with performance up to 15 times faster than Spark and Flink. In this release we have been working on bringing Hazelcast’s legendary programming simplicity to Jet, which we think we have now achieved with the Pipeline API. Programmers, start your Jet engines.”

Also new is fault tolerance using distributed in-memory snapshots - in Hazelcast Jet 0.5 snapshots are distributed across the cluster and held in multiple replicas to provide redundancy. Jet is now able to tolerate multiple faults such as node failure, network partition or job execution failure. Snapshots are periodically created and backed up. If there is a node failure Jet uses the latest state snapshot and automatically restarts all jobs that contain the failed node as a job participant. No additional infrastructure, such as distributed file system or external snapshot storage, is necessary to ensure Hazelcast Jet is fault tolerant out of the box.