allegro is Polish biggest ecommerce site. To sustain our growth, we came to the point when technological revolution could no longer wait. Thus our migration from monolithic to microservice architecture.
Soon it became apparent that there are no microservices without async communication and messaging. This is when project Hermes first surfaced. We decided to build it on top of Kafka for its superb performance and fault tolerance. However we were looking for a tool that would be easy to integrate with, regardless of technology. It should also take care of corner cases and things that require expert knowledge of Kafka design.
Hermes helps us integrate our 700+ microservices, at the same time being strongly focused on losing no message. During our time serving production traffic, we encountered a lots of corner cases and situations, where it was hard to keep Kafka cluster stable. Hermes with it's 100% message delivery guarantees had to acquire features, that allow it on accepting traffic even when Kafka is down.
This is how Hermes became the heart of our new architecture, serving 35 000 rps of incoming and over 50 000 rps of outgoing traffic. Integrating Java, Python, Go and PHP. Allowing our developers to manage their own topics and subscriptions. Making it easy to perform message retransmissions and providing tools for debugging every single message.