Impact of Updating Dataset Description via UI, Redshift, or Python SDK

ingestion
1

Original Slack Thread

<@U06TM7M40RK> If I update dataset description via UI, and then update the description in the actual dataset in redshift for example, does it replaces the description updated from UI ?

2

It seems likely that when you update a dataset description via the UI in DataHub, this change is stored separately from the technical metadata ingested from sources like Redshift. According to <Slack Slack discussion>, UI edits are kept isolated from ingested technical descriptions. Human-edited descriptions are shown if they exist, and when you click the edit button, you can see the original description if it was overwritten before.

Therefore, if you update the description in Redshift and re-ingest the metadata into DataHub, the UI-edited description should remain unless explicitly overwritten. The original description from Redshift would not automatically replace the UI-edited description.

Would you like more detailed steps or further assistance on any specific part of this process?

Sources:

  • <Slack discussion on UI and ingested descriptions> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.
3

<@U06TM7M40RK> what about if the description is updated via python sdf, will the succeeding redshift ingestion replaces the description from api ?

4

It seems likely that when you update a dataset description via the Python SDK, this change is stored in the EditableSchemaMetadata aspect. According to <Slack Slack discussion>, UI edits and API updates are kept separate from the technical metadata ingested from sources like Redshift. The human-edited or API-updated descriptions are prioritized and shown if they exist.

Therefore, if you update the description using the Python SDK and then re-ingest metadata from Redshift, the description updated via the API should remain unless explicitly overwritten. The original description from Redshift would not automatically replace the API-updated description.

Would you like more detailed steps or further assistance on any specific part of this process?

Sources:

  • <Slack Slack discussion> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.
5

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><