Troubleshooting MSSQL Server Ingestion Connection Issue

ingestion
1

Original Slack Thread

hello, I am trying to ingest an MSSQL Server source, my SQL Server is installed on my local machine, over localhost:1433, but I keep getting connection refused over the datahub UI

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: _Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

To help you effectively with your MSSQL Server ingestion issue, please provide the following information:

  1. Are you using the UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g., 0.13.1)?
  3. What data source(s) are you integrating with DataHub (e.g., MSSQL, BigQuery)?

Based on the information provided and the data sources available, here are some potential troubleshooting steps:

  1. Verify Ingestion Configuration: Ensure that your ingestion configuration is correctly set up. Here is an example configuration for MSSQL ingestion:

    source:
  type: mssql
  config:
    host_port: "host.docker.internal:1433"
    database: "AdventureWorksLT2019"
    username: "datahub_test"
    password: "${mssql_password}"

  2. Check Network Configuration: Since you mentioned that you can connect via SQL Management Studio and Python console, but not through the UI, ensure that the network configuration allows connections from the DataHub Docker container to your SQL Server. Using host.docker.internal can help bridge the connection between Docker and the host machine.

  3. Inspect Ingestion Logs: Review the logs generated during the ingestion process. You can access these logs through the DataHub UI by clicking on the ingestion run details. Look for any warnings or errors that might indicate why the connection is failing.

  4. Ensure Correct Configuration: Double-check your configuration file to ensure all necessary parameters are correctly set. Incorrect configurations can lead to successful ingestion without the data being properly displayed in the UI.

Would you like more detailed steps on any of these troubleshooting points or further assistance on a specific part of the process?

Sources:

4

UI
version 0.14.0.4
MSSQL

5

<@U07KVU2DYQM>
If you’re using managed ingestion, the ingestion process runs within a container and won’t be able to connect to services running on your host. You’ll need to configure Docker networking to enable this connection.