For whatever reason I was listening to Rob Pike talk about Concurrency is not Parallelism yet again. I had also been using Kafka for the first time recently. It triggered an epiphany.

I first heard about Kafka around six or seven years ago. I asked what it was and I was told of its wonders and how it would solve all my problems and make everything better and how any app not written around Kafka was inferior and it is the future and blah blah blah. I immediately disregarded it all and ignored it.

Having now used it, I can say for certain that I don't know what the fuss is about. I'd never choose to use it again. I'd never design an app around it. I would, instead, strive for simplicity and solve the types of problems which Kafka solves a different way, preferably by avoiding them altogether.

Now, while listening to Rob Pike talk about concurrency it occurred to me one possibility why Kafka was so beloved by those who espouse it. It is a JVM project and far more heavily used in that circle than outside of it. JVM folk are very likely used to a certain way of doing things and building around Kafka entirely breaks those ways. It shakes things up, possibly in a good way. Kafka creates a communication boundary that wouldn't have otherwise existed in a monolithic JVM app. Kafka creates the need for multiple processes doing smaller tasks utilizing varying Kafka features, rather than a single monolith. The requirement of these structures could be a huge benefit and explains the sentiment which I heard all of those years ago.

I think I've learned a bit thinking about these things...

... or I'm completely full of shit.