6 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
This article explains the new asynchronous instant time generation feature in Apache Hudi 1.1 for Flink writers, which allows for non-blocking requests for new instants. This improvement enhances throughput by enabling writers to continue processing without waiting for previous transactions to complete. It also outlines how this feature interacts with Hudi’s file slicing and timeline management.
If you do, here's more
Apache Hudi 1.1 introduces an asynchronous instant generation mechanism for Flink writers, allowing them to request new instant times without waiting for previous commits to complete. Prior to this release, writers were blocked during the commit process, leading to throughput fluctuations in large-scale workloads. The new approach streamlines data ingestion by enabling writers to generate a new instant time while a previous one is still being committed, resulting in a smoother and more stable ingestion process.
Hudi's architecture relies on a timeline that records all operations through monotonically increasing instant times. Each commit is linked to an instant time, which facilitates efficient write rollbacks, file slicing, and incremental queries. In version 1.0, Hudi introduced a file slicing model based on completion time, allowing compaction scheduling to be decoupled from the ingestion process. This flexibility is crucial in concurrent write scenarios, where previous methods risked data loss by enforcing strict ordering between writes and compactions.
To maintain time monotonicity across distributed systems, Hudi employs a TrueTime API, inspired by Google Spanner. This API ensures that timestamps for instants are generated globally without conflicts, solving issues like clock skew and network latency. Under the new asynchronous model, writers can continue to operate efficiently without being blocked, significantly enhancing performance in streaming ingestion scenarios.
Questions about this article
No questions yet.