Deleting Metadata from a Specific Platform Instance using DataHub CLI

ingestion
1

Original Slack Thread

Is there a way using the datahub delete CLI to delete metadata from particular platform instance? I know there are options to specify env, entity-type, platform and urn…but we have a use case to delete data from a platform instance within a platform
<@U01GZEETMEZ> <@U0445MUD81W> ?

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a :thumbsup: or :thumbsdown: so other folks in the Community know if you’re still looking for help on the topic!

3

To delete metadata from a particular platform instance using the DataHub CLI, you can use the --urn option combined with the --recursive flag. This allows you to target a specific platform instance within a platform and delete all contained entities.

Here is a step-by-step guide:

  1. Identify the URN: First, you need to identify the URN of the platform instance you want to delete. This can be found in the DataHub UI or through your ingestion configurations.

  2. Use the datahub delete Command: Use the datahub delete command with the --urn option to specify the platform instance URN and the --recursive flag to delete all contained entities.

Example Command

datahub delete --urn 'urn:li:dataPlatformInstance:(urn:li:dataPlatform:mysql,my_instance_name)' --recursive

This command will recursively delete all metadata associated with the specified platform instance.

Important Considerations

  • Soft vs. Hard Delete: By default, the delete command performs a soft delete. You can add the --hard flag to perform a hard delete if needed.
  • Dry Run: Use the --dry-run flag to see what entities will be deleted before actually performing the delete operation.
  • Force: The --force flag can be used to skip confirmation prompts.

Example with Dry Run and Force

datahub delete --urn 'urn:li:dataPlatformInstance:(urn:li:dataPlatform:mysql,my_instance_name)' --recursive --dry-run --force

This command will show what entities will be deleted without actually performing the deletion and will skip any confirmation prompts.

Would you like more detailed information on how to perform this operation safely, or do you have any specific concerns about the deletion process?

Sources:

4

Please try above solution and let me know if it solved your issue