If you’re on a Mac, among the many ways to install Apache Kafka and the Confluent Platform, there is Homebrew.
The Confluent Platform is available there and installing it is as easy as
brew install confluent-platform.
Unfortunately, once you’ve run that command, all the Confluent components are installed on your machine but it’s not ready to start and there’s not much documentation to help you.
The main advantages with using Homebrew to manage your Confluent packages is that you can have multiple versions of Confluent/Kafka installed and easily switch between them and it uses a lot less resources than Docker.
Because of my work, I constantly have to alter my Kafka setup and try new things, so in this article, I’ll focus on starting and configuring things manually.
To install, run:
Once this is done, your install is not fully functional. You also have to create the appropriate
Then you’re ready to start a zookeeper and a broker.
If you want to modify the configuration first, edit those files:
For the broker:
/usr/local/etc/kafka points to
/usr/local/Cellar/confluent-platform/5.3.1/etc/kafka/ if you’re using 5.3.1.
When you’re happy with the config files, open up a bunch of terminal windows and run the various components:
Then a broker:
For a Connect in distributed mode:
Of course, in addition to the above, all the tools of the platform are installed:
You can use them locally, for example:
If you want more than one zookeeper and one broker (which is enough for many local development tasks or configuration tests), you have to duplicate the configuration files, alter them (at least, add a
broker-id into the broker config and point the brokers to separate
log.dirs folders) and run the above commands into additional terminal windows pointing to separate config files.
And, yes, you’ll end up with many terminal windows, but who cares.