Troubleshooting CLI Ingestion Errors and Compatibility with dbt-Core 1.7

ingestion
1

Original Slack Thread

getting error on CLI ingestion (DataHub CLI version: 0.13.1.3), what does it mean?
ERROR {datahub.entrypoints:205} - Command failed: Failed to connect to DataHub: No connection adapters were found for 'localhost:8080/config'

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
3

Ingest dbt source local file with target_platform: BigQuery

4

Make sure to add the http:// prefix to the server url

5

let me try

6

Now I got this error:

...
settings = RsTokenizerSettings(
TypeError: TokenizerSettings.__new__() got an unexpected keyword argument 'heredoc_tag_is_identifier'```
7

Huh what acryl-datahub version is that with?

8

acryl-datahub 0.13.1.3

9

what does pip freeze show?

10

e.g. the full dependency list?

11 
acryl-datahub-classify==0.0.10
acryl-sqlglot==23.2.1.dev5```
12

I am running datahub ingest from a docker container with Python 3.10.14.

13

Do you have other python packages installed in the same environment?

14

Specifically, my guess is that sqlglotrs is installed and is causing issues

15 
SQLAlchemy==1.4.52
sqlalchemy-bigquery==1.11.0
sqlglot @ git+<https://github.com/mage-ai/sqlglot@8770e3b7855110a82cb3bc05f3cb6c36a88cfdb2>
sqlglotrs==0.1.0
sqlparams==6.0.1
sqlparse==0.4.3```
16

yup looks like that’s it. you can work around this by using pip install acryl-sqlglot[rs], or by setting the env variable
SQLGLOTRS_TOKENIZER=0

17

btw, the acryl-sqlglot package and the sqlglot version from mage-ai might conflict - I’d recommend using python virtualenvs to avoid this

18

ok

19

Both SQLGLOTRS_TOKENIZER=0 and venv works. Thanks for your help

20

Does the datahub ingest support dbt-core 1.7? It seems to have trouble parsing dbt v1.7’s manifest.json.