Disabling Redshift Automatic Lineage in DataHub Configuration

Original Slack Thread

Hello Everyone,
Just wanted to check is there any way to disable automated lineages created in Redshift in DataHub.

Yes, table lineage, view lineage and column-level lineage are all configurable within the provided redshift integration:

Hello <@U05JJ9WESHL>, Thanks for the reply.
Could you please highlight which specific option will help me in disabling Redshift automatic lineage created with views and how can we disable it.

I have tried to exclude “include_table_lineage” option but its still creating the automatic lineage. Also have tried all the below 3 lineage collector types :-
But its still not working.

Also I have tried to exclude “include_view_lineage” option in YAML recipe as below but its still not working. I am getting the error as "Failed to configure Redshift.
Extra fields not permitted “include view lineage”.

Kindly suggest if there is any specific thing which is required to be performed.attachment

Extra fields not permitted "include view lineage". - I usually get this error when I accidentally add a field outside of the config section.

After clearing your current redshift data with the CLI, Try the following:

    type: redshift
        include_table_lineage: false
        include_table_location_lineage: false
        extract_column_level_lineage: false
        include_unload_lineage: false
        include_view_lineage: false
        include_view_column_lineage: false
        include_copy_lineage: false

Hello <@U05JJ9WESHL>,

Thanks for the help, I have tried to delete the complete Redshift data and included the above recipe in the config section and then run the ingestion still I am getting the below error.

  extra fields not permitted (type=value_error.extra)
  extra fields not permitted (type=value_error.extra)
  extra fields not permitted (type=value_error.extra)```

hmm, the docs may need an update then. With those removed, does the view lineage still render?
To me, the code reads like having include_table_lineage and include_copy_lineage set to false will skip all lineage, even views:

So I would also set include_copy_lineage to false :slightly_smiling_face: . Missed that one.

Thank you so much <@U05JJ9WESHL>! This solution worked for me.

Nice! Glad we were able to figure this one out :+1: