Resolving Ingestion Configuration Issues in DataHub and Great Expectations Integration

troubleshoot
1

Original Slack Thread

(‘Unable to emit metadata to DataHub GMS’, {‘message’: '404 Client Error: Not Found for url: http://localhost:9002/aspects?action=ingestProposal’})
PS C:\Users\damico\OneDrive - http://darlingconsulting.com|darlingconsulting.com\CODE\greatExpectations> datahub --version
acryl-datahub, version 0.13.1.3

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
3

Generally port 9002 is used by the frontend. It seems to me that there is something wrong with your ingestion configuration. Are you running using the CLI? If this is the case, try changing the host to http://localhost:8080

4

using 8080 prevented the error from presenting itself! Thank You! being so new to datahub, i do not know how to see the results in the validation tab on the datahub ui

5

I have never used GE ingestion, but I believe the results will appear in the “validation” tab in datasets. There are some things that depend. If you are ingesting validation results from PostgreSQL tables, for example, it would be interesting for you to also ingest the PostgreSQL metadata as well. The ingest of GE I believe only ingests temporal aspects. If there is no non-temporal metadata registered in the datahub, I believe it should not appear in the UI.

6

Thanks Patrick. Yeah, it appears to be upserting but the validation tab stays grayed out. i have been changing the “platform_instance_map”: {“DDW_1st & Constitution Bank DAA”: “curIntRate”} setting but to no avail :disappointed: curIntRate is the column. I have tried table name, database name and platform name (SQLServer) but haven’t got it yet. I’ll keep plugging away at it! The key to get me going though was the port - so thanks you so much for that!