Message delivery reliability

There are a few message delivery semantics on which message delivery guarantees (or lack of them) are based:

  • At most one delivery: This means the message will be delivered zero or one times. It means the message may be lost. It's the cheapest way and is also known as fire and forget.
  • At least one delivery: This means in multiple delivery attempts, there may be a chance that the message may be duplicated, but it will not be lost for sure.
  • Exactly one delivery: This means that the message is only delivered once. It's neither getting lost or duplicated. 

Structured Streaming provides an end-to-end and exactly one message delivery guarantee based on the following features:

  • Offset tracking in WAL
  • State management (using the in-memory state and WAL)
  • Fault-tolerant sources and sinks (using WAL)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset