Troubleshooting Datahub Errors and Solutions

troubleshoot
1

Original Slack Thread

hello, i am trying datahub out and receiving a few errors.

I have set datahub up on my M1 device as well as a windows laptop. for both devices, I have used either the docker-compose file that comes with running the quickstart command or the quickstart command itself. the former is because i would like to connect to dagster as well.

i am receiving the following errors on both devices for both methods of running datahub

  1. broker is connecting to the wrong cluster ID. this happens when i stop my containers and attempt to start them up again. to solve it, i usually run docker volume prune and it solves the problem, but usually leaves me with no record of my data on datahub/data i’ve ingested, users i’ve created. it’s also a huge hassle
broker                            |     at kafka.server.KafkaServer.startup(KafkaServer.scala:234)
broker                            |     at kafka.Kafka$.main(Kafka.scala:115)
broker                            |     at kafka.Kafka.main(Kafka.scala)
broker                            | [2023-08-22 15:51:59,464] INFO shutting down (kafka.server.KafkaServer)
broker exited with code 1
dependency failed to start: container for service "broker" exited (1)```
2. container mysql is unhealthy. on docker, the container is actually green and it says that it is running. i can also see tables being created. but my docker-compose command still exits out of the terminal. this kind of happens sporadically and i usually just shut down my containers and start them up again and hope for the best. but because of this, i have not been able to meaningfully explore datahub’s features
```kafka-setup                       | DATAHUB_PRECREATE_TOPICS=false
kafka-setup                       | Pre-creation of topics has been turned off, exiting
dependency failed to start: container for service "mysql" is unhealthy```
hoping for some help, this has been more time consuming than i expected :sweat:
2

Hey <@U05NSFH8T6X>, I was just having this same error and spent a ton of time trying to troubleshoot. What i ended up doing to get it to work was to just completely remove everything and start over. I know that’s not exactly ideal (as you noted) but I couldn’t get the containers into a healthy state unless I just nuked it all

3

docker system prune
datahub docker nuke

4

Hi <@U05C47JN04S>, I believe I’ve pinpointed the issue as well. I edited the docker-compose file to make the health check a bit less aggressive for mysql and it seems to have fixed the issue (^ i mentioned in the original post that the containers actually do seem healthy). Prior to that I pruned and nuked many times without seeing results — it might be my internet speed that’s causing the original health checks to fail. If you are still experiencing it, hope this works for you too!

** This solution will not work for the datahub docker quickstart command because there’s no way to edit the configuration there

5

<@U05NSFH8T6X> <@U05C47JN04S>: datahub docker quickstart supports an option to pass in a docker compose file, datahub docker quickstart -f &lt;path_to_docker_compose_file&gt;