Troubleshooting Lineage Information Fetching Issue in DataHub UI

Original Slack Thread

Hey team! I have updated my datahub server to version 0.14.0.2 but getting issue with frontend UI basically in fetching lineage information. I am attaching error SS below:
attachment

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?


:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses! We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a :thumbsup: or :thumbsdown: so other folks in the Community know if you’re still looking for help on the topic and to help us improve the assistant!

To help you more effectively, could you please provide the following information?

  1. Are you using the UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g., 0.14.0.2)?
  3. What data source(s) are you integrating with DataHub (e.g., BigQuery)?

Additionally, based on the information provided, here are a few potential areas to check:

  1. Dataset Existence: Ensure that the upstream and downstream datasets exist in DataHub. According to <Slack Slack thread>, lineage information may not be displayed if the datasets do not exist in DataHub.

  2. Lineage Extraction Support: Verify that the data source plugin supports the specific processors you are using. As per the <https://datahubproject.io/docs/0.13.1/generated/ingestion/sources/nifi|NiFi documentation>, there are limitations on the processors supported for lineage extraction.

  3. Manual Lineage Addition: If automatic lineage extraction is not working, you can manually add lineage using the DataHub UI or APIs. This is mentioned in the <https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/scripts/docgen.py|lineage feature guide>.

Please provide the requested details so we can assist you further.

Sources:

Using UI for the ingestion with version 0.14.0.2

I am introspecting for snowflake data source via UI , getting all the metadata correctly but lineage tab is throwing error i.e. Failed to load

Hey <@U03T38RMG6N> couple things you might be able to try below.

  1. Check for Errors in Logs:
    Look for any specific error messages in the DataHub logs that might give more insight into why the lineage tab is failing to load. You can check the logs from the GMS (Graph Metadata Service) and the frontend service.

  2. Network Tab in Browser:
    Open the network tab in your browser’s developer tools and look for the GraphQL query that is failing. The error message in the response payload can provide more details about the issue.

Let me know if this is helpful