Driven by the need for streaming data in the world of microservices and cloud deployment where new components must interact with legacy systems, project Alpakka was born!
Alpakka Kafka is an open source initiative to implement stream-aware, reactive, integration pipelines for Java and Scala.
According to the GitHub repo, it is built on top of Akka Streams and has been designed from the ground up to understand streaming natively and provide a DSL for reactive and stream-oriented programming, with built-in support for backpressure.
Today, we take a closer look at what Alpakka Kafka has to offer and we’ll go over the important highlights featured in its first milestone release.
The features
Here are the main features Alpakka Kafka has to offer:
- Provides Apache Kafka connectivity for Akka Streams
- Supports consuming messages from Kafka into Akka Streams with at-most-once, at-least-once and transactional semantics, and supports producing messages to Kafka
- Once consumed messages are in the Akka Stream, the whole flexibility of all Akka Stream operators becomes available
- achieves back-pressure for consuming by automatically pausing and resuming its Kafka subscriptions
- Alpakka Kafka 1.0 uses the Apache Kafka Java client 2.1.0 internally
The 1.0 release also features some important changes since 0.22:
- Upgrade to Kafka client 2.1.0 #660 – This upgrade makes it possible to use of the zstandard compression (with Kafka 2.1 brokers). Use Kafka client 2.x poll API #614.
- No more
WakeupException
– The Kafka client API 2.x allows for specifying a timeout when polling the Kafka broker, thus you do not need to use the cranky tool of Kafka’sWakeupException
s to be sure not to block a previous thread. The settings to configure wake-ups are not used anymore. - Alpakka Kafka consumers don’t fail for non-responding Kafka brokers anymore (as they used to after a number of
WakeupException
s). - New
Committer.sink
andCommitter.flow
for standardized committing #622 and #644 - Commit with metadata #563 and #579
- Java APIs for all settings classes #616
- Alpakka Kafka testkit
Head over to the official documentation or the GitHub repo to find all the relevant information on Alpakka Kafka and get started in no time!
If you are interested in learning more about Akka, don’t miss Manuel Bernhardt’s “Tour of Akka Cluster” and “Akka anti-patterns” collections:
The post Implement stream-aware, reactive, integration pipelines with Alpakka Kafka appeared first on JAXenter.
Source : JAXenter