Troubleshooting dbt Core Ingestion Issues via DataHub Quickstart

ingestion
1

Original Slack Thread

Hi, I’m having trouble with ingesting dbt core. I’m currently using the DataHub quickstart, and when I try to use the YAML to build out a dbt-core source, it says that I need to pip install acryl-datahub[dbt]. I tried to exec into the containers running via docker-compose, but the one that allowed me to pip install said that it’s already installed.

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
3

Using UI for ingestion.

4

Using the latest version of DataHub in the docs for quickstart.

5

We have two methods for doing ingestion - UI-ingestion and CLI ingestion: https://datahubproject.io/docs/next/ui-ingestion https://datahubproject.io/docs/next/metadata-ingestion/cli-ingestion

For dbt, if you have manifest/catalog files that aren’t on s3, I’d recommend using cli ingestion

6

Thanks! That worked.