allegro is Polish biggest ecommerce site. To sustain our growth, we came to the point when technological revolution could no longer wait. Thus, our migration from monolithic to microservice architecture.
Soon it became apparent that there are no microservices without async communication and messaging. This is when project Hermes first surfaced. We decided to build it on top of Kafka for its superb performance and fault tolerance. However, we were looking for a tool that would be easy to integrate with, regardless of technology. It should also take care of corner cases and things that require expert knowledge of Kafka design.
Hermes helps us integrate our 2000+ microservices, at the same time being strongly focused on losing no message. During our time-serving production traffic, we encountered a lots of corner cases and situations, where it was hard to keep Kafka cluster stable. Hermes with it's 100% message delivery guarantees had to acquire features, that allow it on accepting traffic even when Kafka is down.
This is how Hermes became the heart of our new architecture, serving 300 000 rps of incoming and over 1 000 000 rps of outgoing traffic. Integrating Java, Python, Go and PHP. Allowing our developers to manage their own topics and subscriptions. Making it easy to perform message retransmissions and providing tools for debugging every single message.