Resolving Duplicate Entries Issue with Acryl-Datahub Redshift Metadata

Original Slack Thread

Hi team, I am using acryl-datahub, version 0.10.5.5.
Using the platform instance configuration, I collected Redshift metadata, but I’m encountering an issue where the same dataset appears as two separate entries.
Within the ‘da’ schema, there are 162 tables. However, they’re displayed as 111 and 51, totaling 162, in the navigation bar. How can I resolve this?

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
  1. using CLI for ingestion
  2. version 0.10.5.5
  3. Redshift

Seems related to this issue https://github.com/datahub-project/datahub/issues/9276

Could you try the workaround and let me know if that works?

I checked the issue link you shared, re-collected the metadata and the issue is resolved. Thank you.