Setting up MySQL and Kafka Clusters in Datahub with Docker Compose Override

Original Slack Thread

Hi Team,
I completed the datahub quickstart using Docker. I would like to use my own MySQL database. Is there an initialization init file in the MySQL database? I found that this file is not in my image

The mysql-setup docker image contains the file. That file will be used against your database when the mysql-setup container is used with your mysql database. There is no need to run that script manually as long as the mysql-setup container is executed pointing to your database.

Okey, Hi <@U03MF8MU5P0>, thanks for your reply.
Actually, you know that we also have MySQL and Kafka clusters in our Database environment. I want to use my own MySQL and Kafka to replace the MySQL and Kafka started by Docker. Therefore, based on your above content, such as MySQL settings, which are used to control connections, I would like to modify the MySQL configuration in the settings section to my own configuration in the docker quickstart.yml file, and then this Docker MySQL doesn’t need to be restarted, right?
Another question is, do I need to initialize tasks such as creating tables and topics when using my own MySQL, Kafka, and other components? If so, can you tell me how to check?

@chengao - If you override the docker compose files to use your down mysql and kafka, then you do not need to run them using docker. Yes, the setup jobs (mysql-setup and kafka-setup) need to execute against your instances. The docker compose configuration assumes that authentication is disabled so you may need to override additional environment variables. The specific variables depends on your specific configuration. Additional documentation for kafka exists here https://datahubproject.io/docs/how/kafka-config and mysql credentials are being used here https://github.com/acryldata/datahub-fork/blob/acryl-main/docker/mysql-setup/init.sh#L16