Troubleshooting hang during dbt ingestion with dbt v1.7 run_results.json and DataHub CLI version: 1!0.13.2+docker

ingestion
1

Original Slack Thread

dbt ingestion hangs with dbt v1.7 run_results.json and DataHub CLI version: 1!0.13.2+docker


[2024-05-14 01:07:14,715] INFO     {datahub.cli.ingest_cli:147} - DataHub CLI version: 1!0.13.2+docker
[2024-05-14 01:07:14,784] INFO     {datahub.ingestion.run.pipeline:239} - Sink configured successfully. DataHubRestEmitter: configured to talk to <http://datahub-datahub-gms:8080> with token: eyJh**********suQs
[2024-05-14 01:07:15,276] INFO     {datahub.ingestion.run.pipeline:256} - Source configured successfully.
[2024-05-14 01:07:15,276] INFO     {datahub.cli.ingest_cli:128} - Starting metadata ingestion
-[2024-05-14 01:07:16,444] WARNING  {sqlglot.optimizer.scope:550} - Cannot traverse scope _u_0."" with type '<class 'sqlglot.expressions.Column'>'
[2024-05-14 01:07:16,448] WARNING  {sqlglot.optimizer.scope:550} - Cannot traverse scope _u_2."" with type '<class 'sqlglot.expressions.Column'>'
|```
2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
3

Can you run with debug logs enabled? It sounds like there’s some sql query that is causing our sql parser to hang, and the logs will help with fixing that