Error in Redshift metadata ingestion to DataHub GMS and recommendation to upgrade to latest version

Original Slack Thread

team, facing below error when ingesting redshift metadata to datahub.

“error”: “Unable to emit metadata to DataHub GMS: Failed to validate record with class com.linkedin.dataset.DatasetProperties: ERROR :: /lastModified :: unrecognized field found but not allowed\nERROR :: /created :: unrecognized field found but not allowed\n”

I can see metadata from table into datahub however ingestion pipeline fails. DataHub is hosted in EKS environment
Any help is appreciated

Could you please share debug log and DataHub version?

DataHub CLI version: 0.11.0
Python version: 3.10.10 (main, Feb 13 2023, 03:34:02) [Clang 14.0.0 (clang-1400.0.29.202)]

Since I am running via UI, couldnt gather the debug log. Here is complete log from UI for reference.


<@U0348BYAS56> added the debug log for the ingestion

server version 0.8.45

It is very old

Please upgrade to latest DataHub

One of my team mate, did upgrade to latest datahub and still issue persists

<@U05RFBBUD5M> Please share the debug log for latest upgraded version