Discussion on using a generic SQL ingester for parsing scripts in DataHub

Original Slack Thread

Hi, is there a generic SQL ingester out there that can read SQL scripts and parse it into datahub and display the lineage?

<@U01GZEETMEZ> might be able to speak to this!

It’s still fairly early/experimental, but we have a source for that https://datahubproject.io/docs/next/generated/ingestion/sources/sql-queries/

The docs aren’t super fleshed out yet, so let me know if you have questions

I tried runnning on ( but seems like it’s not ready yet? got the error Failed to find a registered source for type sql-queries: ‘Did not find a registered class for sql-queries’. when I try pip I get WARNING: acryl-datahub does not provide the extra ‘sql-queries’ . Is it because it is still in experimental mode? When can we expect this to be released?

It should be available in 0.11.0