Azure Event Hubs is a log-based messaging system-as-a-service in Azure cloud. It’s designed to be able to handle huge amount of data, and naturally supports multiple consumers. While Event Hubs are formally part of Azure Service Bus family of products, in fact its model is quite different. Each consumer can peek messages from the queue, do the required processing and then complete the message to remove it from the queue, or abort the processing. Event Hubs service is different. Each Hub represnts a log of messages. And the only operation that is supported for consumers is “give me some messages starting at the offset X”. Microsoft provides the consumer API called EventProcessorHost. This API has an implementation of consumers based on checkpointing. Any failures are ignored: there’s no retry or Dead Letter Queue. In this post I want to focus on a way to process events with higher consistency requirements.
