Resolving MySQL Dependency Error After Running DataHub Docker Quickstart

ingestion
1

Original Slack Thread

<@U06TM7M40RK> how to resolve my-sql dependency error after running “datahub docker quickstart” while setting up datahub in local

2

To resolve MySQL dependency errors after running datahub docker quickstart while setting up DataHub locally, you can follow these steps:

  1. Check for Port Conflicts: One common issue is that the default MySQL port (3306) is already in use by another process on your system. You can either stop the process using this port or change the port for the MySQL container.

    • Stop the Process Using Port 3306:

      sudo lsof -i :3306
kill -15 <PID>

    • Change the MySQL Port: You can override the default port by passing an additional flag to the datahub docker quickstart command.

      datahub docker quickstart --mysql-port 53306

    According to the <Quickstart Debugging Guide | DataHub Debugging Guide>, this should help resolve port conflicts.

  2. Ensure Proper Folder Structure: Make sure the MySQL directory is correctly placed. As per the <Slack discussion>, the MySQL directory should be outside the plugins directory.

  3. Prune Docker System: Sometimes, Docker state issues can cause problems. Pruning the Docker system can help resolve these issues.

    docker system prune

  4. Check for Other Common Issues: Refer to the <https://datahubproject.io/docs/0.13.1/troubleshooting/quickstart|Quickstart Debugging Guide> for other common issues such as OCI runtime create failed, failed to register layer, and toomanyrequests.

  5. Re-run Quickstart: After addressing the above issues, re-run the datahub docker quickstart command.

If you need more detailed steps or run into specific errors, please let me know!

Sources:

3

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a :thumbsup: or :thumbsdown: so other folks in the Community know if you’re still looking for help on the topic!

4

To help you resolve the MySQL dependency error after running datahub docker quickstart, I need a bit more information:

  1. Are you using the UI or CLI for ingestion?
  2. Which DataHub version are you using (e.g., 0.12.0)?
  3. What data source(s) are you integrating with DataHub (e.g., BigQuery)?

In the meantime, here are some common solutions for MySQL-related issues during the DataHub setup:

Common Solutions

  1. Port Conflict:
    If you encounter an error like bind: address already in use, it means the MySQL port (3306) is already in use. You can either stop the service using the port or change the port number for the Docker container.

    sudo lsof -i :3306
kill -15 <PID found in step1>

    Alternatively, you can override the MySQL port by passing an additional flag to the datahub docker quickstart command:

    datahub docker quickstart --mysql-port 53306

    More details can be found https://github.com/datahub-project/datahub/blob/master/docs/troubleshooting/quickstart.md|here.

  2. MySQL Service Already Running:
    If MySQL is already running on your system, you might need to stop it before running the quickstart command.

    service mysql stop
datahub docker quickstart

    This issue is discussed https://github.com/datahub-project/datahub/issues/4278|here.

  3. Docker Issues:
    Sometimes, Docker-related issues can cause problems. Pruning your Docker state can help resolve these issues.

    docker system prune

    For more Docker-related troubleshooting, refer to <https://datahubproject.io/docs/0.13.1/troubleshooting/quickstart|this guide>.

  4. Compatibility Issues:
    If you are using an Apple M1 chip, you might face compatibility issues with the default MySQL image. You can customize the Docker Compose file to use a different MySQL version.

    datahub docker quickstart --arch m1

    More information on customizing the installation can be found https://datahubproject.io/docs/quickstart/#customize-installation|here.

Would you like more detailed steps on any of these solutions?

Sources: