Ingesting Snowflake for Higher Quality Column Lineage in dbt

ingestion
1

Original Slack Thread

Hi All, I have a stupid question here, I have read a few threads on this but still a little fuzzy.

I ingest dbt->snowflake using the cli based file ingestion from dbt (no current snowflake ingestion) - (manifest.json, catalog.json, run_results.json) - yaml below:.
I can see the UI combines the dbt model and snowflake table/view together. Obviously I can see the table lineage, but no column level lineage. Do I need to ingest snowflake as well to get that, or is this not available yet through dbt. All of our snowflake tables / views are created by dbt so I would think I would get it from dbt.

Any thoughts or direction?

    type: datahub-rest
    config:
      server: ${DATAHUB_GMS_URL}
      token: ${DATAHUB_GMS_TOKEN}
source:
  type: "dbt"
  config:
    # Coordinates
    # To use this as-is, set the environment variable DBT_PROJECT_ROOT to the root folder of your dbt project
    manifest_path: "./artifacts/manifest.json"
    catalog_path: "./artifacts/catalog.json"
    test_results_path: "./artifacts/run_results.json"

    # Options
    target_platform: "snowflake" # e.g. bigquery/postgres/etc.```
2

Yup you’ll also need to ingest snowflake - we get higher quality column lineage from snowflake, whereas getting it from dbt requires sql parsing. So that’s why we prefer getting it from snowflake