Building Regex Expression to Exclude `ml__` Views in DataHub Ingestion Recipe

ingestion
1

Original Slack Thread

<@U06TM7M40RK> help me build a regex expression to exclude views beginning with ml__ and show me an example in a recipe

2

To exclude views beginning with ml__ in a DataHub ingestion recipe, you can use a regular expression (regex) pattern. The regex pattern to match views starting with ml__ would be ^ml__.*.

Here’s an example of how you can incorporate this regex pattern into a DataHub ingestion recipe to exclude such views:

source:
  type: your_source_type
  config:
    view_pattern:
      deny:
        - "^ml__.*"

In this example:

  • view_pattern.deny is used to specify the regex patterns for views to exclude from ingestion.
  • The regex pattern ^ml__.* matches any view name that starts with ml__.

This configuration ensures that any view name starting with ml__ will be excluded from the ingestion process.

Would you like more details on this configuration or any other aspect of DataHub ingestion recipes?

Sources: