Apache Kafka 2.3 arrives with improved monitoring for partitions which have lost replicas

Apache Kafka 2.3 is here!

This version brings a long list of important improvements and new features including improved monitoring for partitions which have lost replicas and the addition of a Maximum Log Compaction Lag, which can help make your applications more GDPR compliant!

Let’s have a look at the most interesting highlights.

What’s new in Apache Kafka 2.3?

KIP-351 and KIP-427: Improved monitoring for partitions which have lost replicas – Adds additional metrics showing partitions that have exactly the minimum number of in-sync replicas. By monitoring these metrics, users can see partitions that are on the verge of becoming under-replicated. What’s more,  KIP-351 adds the –under-min-isr command line flag to the kafka-topics command. This allows users to easily see which topics have fewer than the minimum number of in-sync replicas.

KIP-354: Add a Maximum Log Compaction Lag – With KIP-354, it’s possible to set the maximum amount of time an old value will stick around. The new parameter max.log.compation.time.ms specifies how long an old value may possibly live in a compacted topic. This can be used in complying with data retention regulations such as the GDPR.

KIP-402: Improve fairness in SocketServer processors – Prioritizes existing connections over new ones, which improves the broker’s resilience to connection storms.

KIP-461: Improve failure handling in the Replica Fetcher – If a single partition managed by a given replica fetcher thread fails, the thread continues handling the remainder of its partitions.

Other improvements and updates include:

• KAFKA-7283: Reduce the amount of time the broker spends scanning log files when starting up
• KIP-415: Incremental Cooperative Rebalancing in Kafka Connect
• KIP-449: Add connector contexts to Connect worker logs
• KIP-258: Allow Users to Store Record Timestamps in RocksDB
• KIP-428: Add in-memory window store/KIP-445: Add in-memory Session Store
• KIP-313: Add KStream.flatTransform and KStream.flatTransformValues

You can find the full list of new features, updates, and improvements for Apache Kafka 2.3 here.

SEE ALSO: Implement stream-aware, reactive, integration pipelines with Alpakka Kafka

You can also check out this informative video by Tim Berglund explaining what’s new in this release.

quwgeoiw

The post Apache Kafka 2.3 arrives with improved monitoring for partitions which have lost replicas appeared first on JAXenter.