Issue with Visualizing Lineage for a Dataset with ~200 Upstreams & ~200 Downstreams causing `java.lang.StackOverflow` Exception

troubleshoot
1

Original Slack Thread

I’m having an issue where visualizing lineage for a dataset with ~200 upstreams & ~200 downstreams is throwing a java.lang.StackOverflow exception. I’m running Datahub 0.13.1.

I see the following behavior:
• The frontend makes a request on $<http://datahub.com/api/v2/graphql|datahub.com/api/v2/graphql>
• If I tail the GMS logs, I see an error like: 2024-04-24 18:30:51,970 [ForkJoinPool.commonPool-worker-396] ERROR c.l.d.g.e.DataHubDataFetcherExceptionHandler:45 - Failed to execute, java.util.concurrent.CompletionException: java.lang.StackOverflowError, and then a ~400k line traceback
• The frontend eventually gets a 503 Service Unavailable
• This leads to a “Something went wrong” page on datahub.
The weird part is that when I make the same graphql request on $<http://datahub-gms.com/api/graphql|datahub-gms.com/api/graphql> and pull the request from the source, it returns in about ~5 seconds successfully.

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
3

I’d try to post the stacktrace, but it’s 56.1MB :confused:

4

actually, here it is![attachment]({‘ID’: ‘F070VN9T13K’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U05FUSCJ80N’, ‘CREATED’: ‘2024-04-24 18:45:19+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-04-24 18:45:19+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘Plain Text’, ‘NAME’: ‘gms stack trace.txt’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘text/plain’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F070VN9T13K-edc804344c’, ‘FILETYPE’: ‘text’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘gms stack trace.txt’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1713984326.582499’, ‘PARENT_MESSAGE_TS’: ‘1713984020.084829’, ‘MESSAGE_CHANNEL_ID’: ‘C029A3M079U’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 36330779, ‘_FIVETRAN_SYNCED’: ‘2024-04-28 12:56:56.716000+00:00’})