Ingesting SAP HANA Data with Specific Schema and Table Filtering

ingestion
1

Original Slack Thread

how to ingestion SAP HANA with allow some special schema, table ?

2

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

To ingest metadata from SAP HANA while allowing only specific schemas and tables, you can use the SQLAlchemy ingestion source in DataHub, which supports a dialect for HANA. This allows you to specify patterns to include or exclude certain schemas and tables during the ingestion process.

  1. Use SQLAlchemy Source: According to a <Slack conversation>, the SQLAlchemy source can be used for HANA, and it supports filtering by schema and table.

  2. Specify Allow/Deny Patterns: You can define regex patterns to allow or deny specific schemas and tables. This is a common feature across various DataHub ingestion sources, as seen in the https://datahubproject.io/docs/0.13.1/generated/ingestion/sources/trino#config-details|Trino and https://datahubproject.io/docs/0.13.1/generated/ingestion/sources/presto#config-details|Presto sources. These patterns are used to filter the data assets that are ingested.

  3. Configuration Example: While the exact configuration for HANA is not explicitly detailed in the provided data sources, you can refer to the general pattern usage in other sources. For instance, you might configure your ingestion recipe to include specific schemas and tables using regex patterns in the schema_pattern and table_pattern fields.

Would you like more detailed guidance on setting up the SQLAlchemy source for HANA, or do you have any specific questions about the configuration?

Sources:

4

Hi, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.