Ingesting Data from Redash Datasources in DataHub

Original Slack Thread

When ingesting from redash, is it possible to read from datasources on the redash instance?

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

It seems likely that you are asking about the capabilities of DataHub’s integration with Redash, specifically regarding the ingestion of data sources. Based on the <https://datahubproject.io/docs/0.14.0/generated/ingestion/sources/redash|DataHub Redash documentation>, the Redash source plugin can extract Redash dashboards and queries/visualizations, and it has the capability to extract Redash chart table lineages, although this feature is disabled by default.

If you are experiencing issues with the ingestion process, it might be helpful to check the configuration options for the Redash source. According to the <https://datahubproject.io/docs/0.14.0/generated/ingestion/sources/redash#code-coordinates|DataHub documentation>, the parse_table_names_from_sql option is disabled by default, but it can be enabled to try parsing the tables the chart depends on. This is done using the sqlglot package, but it’s important to note that this package doesn’t officially support all SQL dialects that Redash supports, so the results might not always be correct. You can implement a custom parser if needed by setting the sql_parser configuration value.

If you are still facing issues, it might be helpful to run the ingestion with the --debug flag to gather more detailed logs, as suggested in the <Slack conversation>. This can provide more insights into what might be going wrong during the ingestion process.

Would you like more detail on any specific part of the process or configuration?

Sources:

Hi, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><