Implementing Tag Propagation Across Datasets, Charts, and Dashboards via Custom Actions

troubleshoot
1

Original Slack Thread

Hi <@UV0M2EB8Q>, I am building a custom action for tag propagation. (I started here with your code: https://github.com/acryldata/datahub-actions/blob/5d881b5c1de2d754b6bb843e261bd2a8e1839e73/datahub-actions/src/datahub_actions/plugin/action/tag/tag_propagation_action.py|https://github.com/acryldata/datahub-actions/blob/5d881b5c1de2d754b6bb843e261bd2a8[…]src/datahub_actions/plugin/action/tag/tag_propagation_action.py). I always like to try to reuse as much as I can, thanks! This code only add tags to datasets. I need the ability to add tags to charts and dashboards as well. Since this code only pulls in the action: config parameters tag_prefixes: from the yaml (see below). The issue I run into is this, I would need to define an emitter via REST, and need the gms_server and token in order to run the emitter, how do I pull in the datahub: information from the yaml in order to use it? The PipelineContext class does not have an emit method that I can use. Thanks for any direction.

  #type: "tag_propagation"
  type: "custom-tag-prop:CustomTagPropagationAction"
  config:
    tag_prefixes:
    - SourcesSDP
datahub:
  server: "https://&lt;my-server&gt;/api/gms"
  token: "&lt;my-token&gt;"```
2

Hey <@U05HMP51HLJ>: there is a graph instance under self.ctx. that you can use to emit metadata

3

e.g. https://github.com/acryldata/datahub-actions/blob/5d881b5c1de2d754b6bb843e261bd2a8e1839e73/datahub-actions/src/datahub_actions/plugin/action/tag/tag_propagation_action.py#L149|https://github.com/acryldata/datahub-actions/blob/5d881b5c1de2d754b6bb843e261bd2a8[…]src/datahub_actions/plugin/action/tag/tag_propagation_action.py

4

if you need the wrapped datahubgraph instance, you can get it via self.ctx.graph.graph

5

also, since I think this would be generally useful… please do contribute this enhancement back!

6

Thanks <@UV0M2EB8Q>. I didn’t think of the graph.graph I will give it a try and look forward to contributing!

7

<@UV0M2EB8Q>, the graph.graph worked and going through some testing. I will do a PR for the change to the tag_propagation.py in the next few days after I test it a little more. Thanks

8

Nice! cc <@U01GZEETMEZ> to keep track of the contrib when it comes through