Troubleshooting SQL Warehouse Issues in Databricks Unity Integration

ingestion
1

Original Slack Thread

<@U06TM7M40RK> Hello, I am trying out the Databricks Unity ingestion but we keep getting following error on SQL warehouse:

ERROR {datahub.entrypoints:199} - Command failed: failed to reach RUNNING, got State.STOPPED: current status: State.STOPPED , can you please suggest how to fix?

2

The error message you’re encountering indicates that the SQL warehouse in Databricks is not reaching the “RUNNING” state and is instead stopping. This can happen due to several reasons, such as resource constraints, configuration issues, or network problems. Here are some steps you can take to troubleshoot and potentially resolve the issue:

  1. Check SQL Warehouse Configuration: Ensure that the SQL warehouse is properly configured with the necessary resources (e.g., cluster size, auto-scaling settings) to handle the workload. You can adjust these settings in the Databricks workspace under the SQL Warehouses tab.

  2. Review Logs: Check the logs for the SQL warehouse in Databricks to identify any specific errors or warnings that might indicate why the warehouse is stopping. This can provide more insight into the underlying issue.

  3. Increase Wait Time: If the warehouse is taking longer to start than expected, you might need to increase the wait time for the warehouse to warm up. Unfortunately, the provided data sources do not specify how to adjust this wait time directly within DataHub, but you can ensure that the warehouse is started and in a “RUNNING” state before initiating the ingestion process.

  4. Network and Permissions: Verify that there are no network connectivity issues between DataHub and the Databricks workspace. Additionally, ensure that the service principal or user has the necessary permissions to access and manage the SQL warehouse.

  5. Databricks Support: If the issue persists, consider reaching out to Databricks support for assistance, as they can provide more detailed insights into the SQL warehouse’s behavior and any potential configuration issues.

Would you like more detailed steps on any of these troubleshooting points or further assistance on a specific part of the process?

Sources:

3

<@U06TM7M40RK> the warehouse used is serverless and auto stop is set to 5 minutes, tested with a manual ingestion, started the sql warehouse first and then start the ingestion.
During the ingestion, the first 30 mins, there’s no activity on the sql warehouse yet, causing the warehouse to auto stop, I enabled it to keep it up during the ingestion.

Result: Ingestion succeeded
how should i do to fix it please?

4

To address the issue of the SQL warehouse stopping due to inactivity during the ingestion process, you can consider the following strategies:

  1. Increase Auto-Stop Timeout: If possible, increase the auto-stop timeout setting for the SQL warehouse to a longer duration. This will prevent the warehouse from stopping due to inactivity during the initial stages of the ingestion process.

  2. Pre-Warm the Warehouse: Before starting the ingestion, manually start the SQL warehouse and run a simple query to keep it active. This can help ensure that the warehouse remains in a “RUNNING” state during the ingestion process.

  3. Monitor and Restart: Implement a monitoring script or process that checks the status of the SQL warehouse during the ingestion. If the warehouse stops, the script can automatically restart it. This can be done using the Databricks API or CLI.

  4. Optimize Ingestion Configuration: Review the ingestion configuration to ensure that it is optimized for performance. This includes checking the number of threads, batch sizes, and any other relevant settings that might affect the speed of the ingestion process.

  5. Use a Dedicated Warehouse: If feasible, consider using a dedicated SQL warehouse for ingestion tasks. This warehouse can be configured with settings that are optimized for ingestion, such as a longer auto-stop timeout and appropriate resource allocation.

  6. Databricks API for Automation: Use the Databricks API to automate the process of starting the SQL warehouse before ingestion and stopping it after the process is complete. This can help ensure that the warehouse is always available when needed.

Would you like more detailed steps on any of these strategies or further assistance with a specific part of the process?

Sources:

5

<@U06TM7M40RK> more detail on Pre-Warm the Warehouse and Optimize Ingestion Configuration:

6

The DataHub’s documentation I have ingested does not mention how to pre-warm the warehouse or optimize ingestion configuration specifically for Databricks SQL warehouses. To provide more detailed guidance, it would be helpful to know the specific configurations you are currently using for your SQL warehouse and any constraints or requirements you have. If this doesn’t help, please reach out to the DataHub support team for further assistance. :+1: 0 button :-1: 0 button :light_bulb: Hint: Mention @runllm in the thread for followups.

7

<@U06TM7M40RK> more ingestion log

 'current_time': '2025-04-17 09:50:46.474151 (now)',
 'total_duration_in_seconds': 10134.59}
[2025-04-17 09:50:46,890] ERROR    {datahub.entrypoints:218} - Command failed: failed to reach RUNNING, got State.STOPPED: current status: State.STOPPED```
it started for a well, but gor the error in the ingestion middle, why?
8

The DataHub’s documentation I have ingested does not mention how to troubleshoot the specific error where the SQL warehouse transitions from “RUNNING” to “STOPPED” during the middle of an ingestion process. To better understand the issue, it would be helpful to know more about the SQL warehouse’s configuration, such as its auto-stop settings, resource allocation, and any relevant logs from the Databricks side that might indicate why the warehouse stopped. If this doesn’t help, please reach out to the DataHub support team for further assistance. :+1: 0 button :-1: 0 button :light_bulb: Hint: Mention @runllm in the thread for followups.

9

<@U06TM7M40RK>
Type Serverless
Cluster size 2X-Large
Auto stop After 5 minutes of inactivity
Scaling Cluster count: Active 0 Min 1 Max 1

10

The DataHub’s documentation I have ingested does not mention how to handle a situation where a serverless SQL warehouse with specific configurations (such as a 2X-Large cluster size and a 5-minute auto-stop setting) stops during an ingestion process. To address this, you might want to consider increasing the auto-stop timeout to prevent the warehouse from stopping due to inactivity, or implementing a script to periodically send a simple query to keep the warehouse active during long ingestion processes. If this doesn’t help, please reach out to the DataHub support team for further assistance. :+1: 0 button :-1: 0 button :light_bulb: Hint: Mention @runllm in the thread for followups.

11

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><