Managing `databricks-sdk` Versions for `ExecuteStatementResponse` Compatibility

ingestion
1

Original Slack Thread

<@U06TM7M40RK> which version of databricks-sdk installed in the virtual environment supports the ExecuteStatementResponse class?

2

The ExecuteStatementResponse class was removed in version 0.30.0 of the databricks-sdk. Therefore, to have support for the ExecuteStatementResponse class, you should use a version of the databricks-sdk that is earlier than 0.30.0. According to the <Issues · datahub-project/datahub · GitHub issue>, downgrading the databricks-sdk to a version below 0.30.0 resolves the issue with the ExecuteStatementResponse class being missing.

Would you like more detailed steps on how to manage the version of databricks-sdk in your environment?

Sources:

3

<@U06TM7M40RK> facing Failed to find a registered source for type unity-catalog: unity-catalog is disabled due to an error in initialization error, even with databricks-sdk 0.29.0 version.

4

The error “Failed to find a registered source for type unity-catalog: unity-catalog is disabled due to an error in initialization” suggests that there might be an issue with the unity-catalog plugin or a dependency conflict. Here are some steps to troubleshoot and potentially resolve the issue:

  1. Verify Plugin Installation: Ensure that the acryl-datahub[unity-catalog] plugin is correctly installed. You can do this by running:

    pip show acryl-datahub[unity-catalog]

  2. Check for Dependency Conflicts: The error might be due to a conflict with the pydantic library or another dependency. Ensure that all dependencies are up-to-date and compatible. Specifically, ensure that pydantic is pinned to a version below 2.0, as pydantic v2 introduces breaking changes.

  3. Reinstall Dependencies: Try reinstalling the dependencies to ensure there are no corrupted installations:

    pip uninstall acryl-datahub[unity-catalog]
pip install 'acryl-datahub[unity-catalog]' 'pydantic<2'

  4. Check DataHub Plugins: Run the following command to check the installed plugins and their statuses:

    datahub check plugins --verbose

  5. Review Configuration: Double-check your ingestion configuration to ensure there are no issues. Refer to the <https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/docs/sources/databricks/unity-catalog_post.md|DataHub Unity Catalog documentation> for the correct setup.

If these steps do not resolve the issue, please provide more details about your setup, such as the DataHub version and any relevant error logs.

Would you like more detailed steps on any of these troubleshooting points or further assistance on a specific part of the process?

Sources:

  • <Slack discussion on plugin issues> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.
5

<@U06TM7M40RK>

shows package not found

6

If the command pip show acryl-datahub[unity-catalog] returns “package not found,” it indicates that the acryl-datahub[unity-catalog] package is not installed in your environment. Here are the steps you can follow to install it:

  1. Upgrade pip, wheel, and setuptools: Ensure you have the latest versions of these tools to avoid any installation issues.

    python3 -m pip install --upgrade pip wheel setuptools

  2. Install the acryl-datahub[unity-catalog] Package: Use the following command to install the package.

    python3 -m pip install 'acryl-datahub[unity-catalog]'

  3. Verify Installation: After installation, verify that the package is installed correctly by running:

    pip show acryl-datahub[unity-catalog]

If you encounter any errors during installation, please provide the error message for further assistance.

Would you like more detailed steps on any of these points or further assistance with the installation process?

Sources:

7

<@U06TM7M40RK> is this power shell compatible commands?

8

The commands provided are compatible with PowerShell, but you may need to adjust the syntax slightly for certain scenarios. Here are the PowerShell-compatible versions of the commands:

  1. Upgrade pip, wheel, and setuptools:

    python -m pip install --upgrade pip wheel setuptools

  2. Install the acryl-datahub[unity-catalog] Package:

    python -m pip install 'acryl-datahub[unity-catalog]'

  3. Verify Installation:

    pip show acryl-datahub[unity-catalog]

These commands should work in a PowerShell environment. If you encounter any issues, please provide the error message for further assistance.

Would you like more detailed steps on any of these points or further assistance with using PowerShell?

If this doesn’t help, please reach out to the DataHub support team for further assistance. :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.

9

<@U06TM7M40RK> which version of Databricks sdk is compatible with ExecuteStatementResponse class?

10

The ExecuteStatementResponse class was removed in version 0.30.0 of the databricks-sdk. Therefore, to have support for the ExecuteStatementResponse class, you should use a version of the databricks-sdk that is earlier than 0.30.0. Specifically, versions up to 0.29.0 should be compatible with the ExecuteStatementResponse class.

Would you like more detailed steps on how to manage the version of databricks-sdk in your environment?

Sources:

11

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><