Managing and Deleting Queries in Snowflake Data Assets

troubleshoot
1

Original Slack Thread

<@U06TM7M40RK> any cli commands to delete the top n queries from snowflake data assets

2

I’m not able to find any data sources to answer your question. You might want to specify if you are looking to delete metadata related to queries or actual data within Snowflake. If this doesn’t help, please reach out to the DataHub support team for further assistance. :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.

3

<@U06TM7M40RK> how to delete the queries from the queries tab

4

It seems likely that deleting queries from the queries tab in DataHub, especially for Snowflake data assets, involves some complexity due to how queries and usage statistics are stored. According to the <https://forum.datahubproject.io/t/managing-queries-and-data-usage-stats-in-snowflake-dataset-on-v0-11-0/365|DataHub forum>, both queries and usage stats are stored in the same underlying “aspect” called datasetUsageStatistics. This makes it challenging to delete queries without affecting usage stats.

To delete both usage stats and queries, you can use the datahub delete CLI command. However, this will remove both queries and usage stats, which might not be desirable if you want to keep the usage stats intact.

Here’s a general approach using the DataHub CLI:

  1. Soft Delete: This will hide the queries but not remove them completely.

    datahub delete --platform snowflake --soft

  2. Hard Delete: This will permanently delete the queries and associated metadata.

    datahub delete --platform snowflake --hard

It’s important to use the --dry-run flag initially to see what would be deleted without actually performing the deletion.

If you want to delete only the queries while keeping the usage stats, you might need to use the API to read the existing data, clear the queries, and then write it back. This process is not straightforward and might require custom scripting.

Would you like more detailed guidance on using the API or any other specific aspect of this process?

Sources: