Troubleshooting Metabase Ingestion Issues in DataHub and Suggestions for Improvement

troubleshoot
1

Original Slack Thread

Hello, I’m setting up ingestion from Metabase (DataHub v0.12.1, UI). It ends being marked as Failed because it can’t read SQL statements sourcing from other questions/models. I have multiple failures like this:
'failures': {'metabase-query': ['Unable to retrieve lineage from query. Query: SELECT SUM(added_and_removed_sum) FROM (\nSELECT SUM(no_of_fields_added) + SUM(no_of_fields_removed) AS added_and_removed_sum FROM {{#1899}}\nUNION\nSELECT COUNT(*) AS type_changed_sum FROM {{#1885}}) Reason: An Identifier is expected, got Token[value: {] instead. '
Is there any way to solve this?
Thanks! :slightly_smiling_face:

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
3

I’ll make a change to mark a SQL parsing failure as a warning, not a failure, so it doesn’t mark the full ingestion run as failed.

4

Great, thanks!
As a separate task, it would be also be great if Metabase could identify this kind of SQL statements in terms of their lineage, effectively tracing the evolution from the original question or model to the new question.