"You're older than I expected" - tricky message retention in Azure Event Hub

Event Hub(or Service Bus if we're talking about a service as "a whole") has a concept of a message retention. In short - you can set a fixed period, which will determine after how many days a message is considered outdated and is no longer available in a bus. So far so good - nothing tricky here and the concept is fairly simple. Unfortunately there're not so obvious gotchas, which can hit you really hard, if you forget about them.

You're older than I expected

The confusion comes from the fact, that Event Hub is a part of a bigger service called Service Bus and is only a subset of available functionalities. When we're considering Service Bus, we're talking about queues, topics and so on. All those have a property called TTL(time-to-live), which is attached to messages being passed. Although TTL means different thing for each different concept(e.g. at-least-once/at-most-once for queues), it's here and its definition is intuitive. The question is - how is this related to message retention mentioned earlier?

The confusion comes from the fact, that different services in Service Bus are designed for different purposes - because of that each treats a definition of message or entity in a slightly different way. Since Event Hubs are considered a big scale solution(in opposite to e.g. queues), they rather track whole blocks of messages rather than a single message, which is being pushed through a pipeline.

This being said, there's a reason why message retention is no always what you can expect - if you're using Event Hub for a fairly small amount of data(tens thousands events per day at most), there's quite a big likelihood, that it won't be considered as "outdated" as long as the container, which holds messages, is full.

I saw this... twice?

Now imagine following situation - you're about to go live with a solution, which is currently on a production environment, and using Event Hub as the heart of it. Let's say Event Hub was gathering data for the last seven days(message retention is set to only one day so this shouldn't be a case) because you wanted to avoid a cold start. Now consumers started and... your clients are receiving events/notifications/pop-ups from a week ago. 

The first problem - you forgot to check in your code whether a message is valid from your point of view. This happens, especially if you consider a documentation as the only source of your information about a service.

The second - well, it was nowhere said, that message retention is not what it looks like at the first glance.

Summary

As you can see, it's a good thing to remodel your way of thinking about messages and entities when working with Event Hub to avoid confusion. Apply a certain level of abstraction to your infrastructure and ask yourself a question - am I working with single messages or maybe whole blobs, which make sense only when are fully processed? If the answer is the former, you can trick yourself sooner or later.

From hate to love - why I moved to VSTS

This is a short post about my journey from hating to really loving VSTS. In fact, for the last few years I was a hardcore fan of TeamCity and really couldn't find any other tool, which would fit my needs. In fact, VSTS was the last thing I'd use to work on a daily basis(mostly because of my bad experience with TFS/VSO). I'll explain what was the reason of changing my mindset.

Learning curve

What surprised my a lot was actually the learning curve of VSTS. I was used to really digging deeper into configuration to actually make magic happen. Here you just grab something and use it. Want a Grunt runner? It's already installed, not need for some 3rd-party stuff. Want to groups some tasks and reuse them? Just go for tasks groups and use them anywhere you want. Also no annoying "default branch" from TeamCity(in pre 10.0 era) is present so working with your git project is a piece of cake.

All in one place

This is something, which I like the most - I have my codebase, builds and releases, all in one place. No more switching between different tools to do a code review, tweak a build and deploy it. It's easy to use, easy to configure and to monitor.

Technology stack

Other tools are perfectly fine when used with .NET platform. But they're not designed for it - and you feel it. Don't get me wrong - I'm aware of the fact, that creating such applications are not a trivial tasks. But after all those years, I've finally come to a conclusion, that having a tool designed specially for your technologies really makes things easier.

Ready for a cloud

The last but not least - since VSTS integrates with Azure seamlessly, I can develop my cloud applications or move from on-prem without much effort. Just use one of many Azure tasks in your build/release and watch how everything is being collected and deployed. No need to configure additional plug-ins - the best thing here is, that once you use it, you'll ask yourself "why other things can't be so easy?".

Those are just a few reasons why I chose VSTS as my main tool for building and managing my applications. I strongly recommend you to give it a chance, even if you disliked it on the first try - especially with recent and upcoming updates, which makes it even better.