Confluent has released an update to its open source Confluent Platform 2.0 which is based on an updated Apache Kafka 0.9 core. The release offers new functionality to enable secure multi-tenant operations, simplify development and maintenance of applications that produce or consume data in Kafka, and provide high-throughput, scalable data integration with a wide array of data sources. The platform turns an organization’s data into readily available low-latency streams, and acts as a buffer between systems that might produce or consume such data at different rates.
The Confluent Platform is built upon
Apache Kafkato provide a central stream data pipeline. Kafka is an open source system for managing real-time streams of data from websites, applications and sensors. Apache Kafka includes pure Java clients (both producers and consumers) that developers can integrate into Java applications or other languages that run on the JVM, including Scala, Clojure, JRuby, and Jython.
Confluent provides client tools for a variety of languages, including Java, C, C++, and allow messages to be produced or consumed with any network connected tool using the company’s REST proxy. Confluent includes a schema registry to help assure data consistency across multiple applications and organizations as well as providing a set of connectors for common sources like databases, files, and Hadoop that make it easy to read data into Kafka, or write data from Kafka.
With Kafka, developers can build a centralized data pipeline enabling microservices or enterprise data integration, such as high-capacity ingestion routes for Apache Hadoop or traditional data warehouses, or as a foundation for advanced stream processing using engines like Apache Spark, Storm or Samza.
Part of this latest release is Kafka Connect, a new connector-driven data integration feature that facilitates large-scale, real-time data import and export for Kafka, enabling developers to integrate various data sources with Kafka without writing code through a 24/7 production environment which includes automatic fault-tolerance, transparent, high-capacity scale-out, and centralized management.
Other new Confluent Platform 2.0 features include:
- Data Encryption: Ensures encryption over the wire using SSL.
- Authentication and Authorization: Allows access control with permissions that can be set on a per-user or per-application basis.
- Quality of Service: Configurable quotas that allow throttling of reads and writes to ensure quality of service.
- New Java consumer: The update includes a new Java consumer simplifying the development, deployment, scalability, and maintenance of consumer applications using Kafka.
- Supported C client: Fully featured producer and consumer implementation.
- Over 500 individual performance, operational, and management improvements.