Ingesting Metadata from Google Sheets using Python SDK in Shared Google Drive

ingestion
1

Original Slack Thread

Hi there;
I am using python to list and describe all the google sheets in a shared google drive.
Using the python sdk, I’d like to ingest metadata about all the files.
Are there any examples covering the entire process?

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)
3

I’m using the Python sdk on the latest version deployed using Docker.

4

We have some docs on building ingestion connectors here https://datahubproject.io/docs/metadata-ingestion/adding-source, and a number of guides linked to here https://datahubproject.io/docs/api/datahub-apis#datahub-api-comparison

5

Thanks <@U01GZEETMEZ> Is there a way to make one of the fields a uri that can be clicked on (and opens in a new tab)?

6

This file doesn’t exist in that location.

The MetadataChangeEventClass is defined in the metadata models which are generated under metadata-ingestion/src/datahub/metadata/schema_classes.py.

7

Those are generated by the build process https://datahubproject.io/docs/metadata-ingestion/developing/#set-up-your-python-environment