Transforming UI with BrowsePathsV2Class in DataHub

ingestion
1

Original Slack Thread

Hello guys! During the ingestion I am using the transformer ‘set_dataset_browse_path’ which should change the hierarchy of the sidebar, but no change are applied. I noticed that this transformer is changing the BrowsePathsClass and not the BrowsePathsV2Class. Can I have some information on how I can use the transform to update also the UI?

2

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

To address your issue with the set_dataset_browse_path transformer not updating the UI, it appears that the transformer is modifying the BrowsePathsClass instead of the BrowsePathsV2Class. The BrowsePathsV2Class is necessary for the updated Browse V2 experience in DataHub.

Steps to Update the UI with BrowsePathsV2Class

  1. Ensure You Are Using the Latest CLI Version: Make sure you are using a CLI version that supports browsePathsV2. According to <Slack Slack conversation>, CLI version v0.10.5 and above should support browsePathsV2.

  2. Custom Transformer for BrowsePathsV2: You may need to write a custom transformer to update the BrowsePathsV2Class. Here is a general outline of how you can achieve this:

Example Custom Transformer

import re
from typing import List

from datahub.configuration.common import ConfigModel
from datahub.ingestion.api.common import PipelineContext
from datahub.ingestion.api.transform import Transformer
from datahub.ingestion.transformer.base_transformer import BaseTransformer, SingleAspectTransformer
from datahub.metadata.schema_classes import MetadataChangeEventClass, BrowsePathsV2Class

class CustomBrowsePathV2TransformerConfig(ConfigModel):
    # Add any configuration parameters your transformer needs here
    pass

class CustomBrowsePathV2Transformer(BaseTransformer, SingleAspectTransformer):
    ctx: PipelineContext
    config: CustomBrowsePathV2TransformerConfig

    def __init__(self, config: CustomBrowsePathV2TransformerConfig, ctx: PipelineContext):
        super().__init__()
        self.ctx = ctx
        self.config = config

    @classmethod
    def create(cls, config_dict: dict, ctx: PipelineContext) -> "CustomBrowsePathV2Transformer":
        config = CustomBrowsePathV2TransformerConfig.parse_obj(config_dict)
        return cls(config, ctx)

    def entity_types(self) -> List[str]:
        return ["dataset"]

    def transform_aspect(self, entity_urn: str, aspect_name: str, aspect: MetadataChangeEventClass) -> MetadataChangeEventClass:
        if aspect_name == "browsePathsV2":
            new_paths = ["/custom/path/for/{}".format(entity_urn)]
            aspect = BrowsePathsV2Class(paths=new_paths)
        return aspect

Integrate the Transformer into Your Ingestion Recipe

Add the custom transformer to your ingestion recipe YAML file:

source:
  type: "your_source_type"
  config:
    # Your source configuration here

transformers:
  - type: "path.to.CustomBrowsePathV2Transformer"
    config:
      # Add any configuration parameters your transformer needs here

sink:
  type: "datahub-rest"
  config:
    server: "http://localhost:8080"
4

Additional Resources

Would you like more detailed steps or a code example on how to implement this?

Sources:

  • <Slack Conversation on BrowsePathsV2> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.
5

It is not a version problem, I am using the version 0.14.0.2 for cli and server. I would like to use a default one, becuase according to the doc it should be already provided

6

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><

7

<@U07FV3UHE4D> did someone test the browse_path transform? It seems not updated

8

Thank you for letting us know you need a person to take a look at this thread.

Either a community member or one of our team members will try to assist when they have a moment, responses unfortunately might not be immediate. We hope this helps speed things along :slight_smile: