Error in Redshift metadata ingestion to DataHub GMS and recommendation to upgrade to latest version

ingestion
1

Original Slack Thread

team, facing below error when ingesting redshift metadata to datahub.

“error”: “Unable to emit metadata to DataHub GMS: Failed to validate record with class com.linkedin.dataset.DatasetProperties: ERROR :: /lastModified :: unrecognized field found but not allowed\nERROR :: /created :: unrecognized field found but not allowed\n”

I can see metadata from table into datahub however ingestion pipeline fails. DataHub is hosted in EKS environment
Any help is appreciated

2

<@U05RFBBUD5M>
Could you please share debug log and DataHub version?

3

DataHub CLI version: 0.11.0
Python version: 3.10.10 (main, Feb 13 2023, 03:34:02) [Clang 14.0.0 (clang-1400.0.29.202)]

Since I am running via UI, couldnt gather the debug log. Here is complete log from UI for reference.

4

attachment

5

<@U0348BYAS56> added the debug log for the ingestion

6

server version 0.8.45

7

It is very old

8

Please upgrade to latest DataHub

9

One of my team mate, did upgrade to latest datahub and still issue persists

10

<@U05RFBBUD5M> Please share the debug log for latest upgraded version