Title: Troubleshooting MSSQL Ingestion Issue with DataHub on Docker

ingestion
1

Original Slack Thread

Hi, I am facing issues while ingesting from mssql server, I am using datahub on docker on my local machine and mssql server is installed on my local machine, can someone please help me

2

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

To help you effectively with your MSSQL ingestion issue using DataHub on Docker, I need a bit more information:

  1. Are you using the UI or CLI for ingestion?
  2. Which version of DataHub are you using?
  3. Can you provide any specific error messages or logs that you are encountering?

In the meantime, here are some common troubleshooting steps for MSSQL ingestion issues:

  1. Network Configuration: Ensure that your MSSQL server is accessible from the Docker container. You might need to use host.docker.internal as the host in your connection string, which allows the Docker container to access services running on the host machine.

  2. Port Exposure: Verify that the MSSQL server’s port (usually 1433) is open and accessible. You can check this by trying to connect to the MSSQL server from another application on your host machine.

  3. Docker Network: Make sure that both DataHub and MSSQL are on the same Docker network if MSSQL is also running in a container. This can be done by creating a custom Docker network and attaching both containers to it.

  4. Recipe Configuration: Ensure that your ingestion recipe is correctly configured. Here is an example configuration for MSSQL ingestion:

    source:
  type: mssql
  config:
    host_port: 'host.docker.internal:1433'
    database: 'your_database'
    username: 'your_username'
    password: 'your_password'
    use_odbc: 'True'
    uri_args:
      driver: 'ODBC Driver 17 for SQL Server'
      Encrypt: 'yes'
      TrustServerCertificate: 'Yes'
      ssl: 'True'

  5. Logs and Debugging: Check the logs for any specific error messages. You can view the logs by clicking on the “Details” button of the failed ingestion run in the UI.

For more detailed steps on debugging ingestion issues, refer to the <Ingestion | DataHub documentation on debugging a failed ingestion run>.

Would you like more detailed guidance on any of these steps?

Sources:

  • <Ingestion | DataHub documentation on debugging a failed ingestion run> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.
4

<@U06TM7M40RK>, facing the following error ‘datahub.ingestion.run.pipeline.PipelineInitError: Failed to configure the source (mssql): No module named ‘pyodbc’’

5

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><