Listing Connection Schemas Before Ingestion using DataHub

ingestion
1

Original Slack Thread

Hello, everyone!
I have a large MySQL database and it is taking 8h to ingest everything (I don`t know the exacts schemas that i’m gonna need beforehand so i’m ingesting everything first). Can I list all the connection schemas before the actual ingestion? Is there a way to do this using DataHub?

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
3

you can use File as Sink to Outputs metadata to a file to verify before actually ingesting metadata to Datahub
https://datahubproject.io/docs/metadata-ingestion/sink_docs/file

4

thanks! I will test that!