Kafka - Getting Started with Kafka

Before you can understand what Kafka is, it's helpful to see how data is transmitted without Kafka. In this extremely simplified diagram, the sending system (producer) transmits data to the receiving system (consumer). In this scenario, the data would most probably be in XML format, and be something like an order placed at an online store.

While this is all fine and dandy, this type of design can produce problems. Most notably, if the receiving system is unable to receive the data (eg. the receving system is down or overwhelmed with requests), the sending system may not be able to move onto the next transmission or may continually try to send the data over and over again (which is a waste of the sending systems resources). Kafka resolves these issues.

 

RabbitMQ sit's between the sending system (producer) and receiving system (consumer), as a sort of broker or proxy. In this way, if the receiving system is unavailable, the sending system does not get trapped in a situation where it cannot move onto it's next transmission or is bogged down trying to send the transmission over and over again. The sending system simply sends the data to Kafka and then moves on (fire and forget). In this way, Kafka makes it so that data tranmission is asynchronous.

 

Let's install Kafka! Then, start Kafka.

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.





Please enter 9c9c8 in the box below so that we can be sure you are a human.




Comments

Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |