Viewing the lineage path between two models in DataHub without manual expansion and guesswork

ui
1

Original Slack Thread

<@U06TM7M40RK> say I know there is a relationship between 2 models, how do I view the path between them without having to guess and expand every up/downstream?

2

To view the path between two models without manually expanding every upstream or downstream relationship, you can utilize DataHub’s lineage features. However, as of the current capabilities, DataHub does not directly support viewing the entire path between two specific entities in a single view without some manual exploration. Here are some steps and considerations based on the available features:

  1. Lineage Tab and Visualization: You can start by using the Lineage tab on the entity page of one of the models. This allows you to see upstream and downstream dependencies. You can toggle between these views to explore the relationships.

  2. Column-Level Lineage: If you are dealing with datasets, DataHub supports column-level lineage, which can help in understanding the specific transformations and data flow between columns of datasets. This might not directly apply to models but is useful for datasets.

  3. GraphQL API: You can use the GraphQL API to query lineage information programmatically. This allows you to specify the direction and degree of relationships you want to explore. For example, you can query for all downstream relationships of a model and then filter the results to find the path to the target model.

  4. Manual Exploration: As a workaround, you might need to manually explore the lineage by expanding nodes in the Lineage Visualization screen. This can be cumbersome if there are many dependencies, but it is currently one of the ways to visually trace the path.

  5. Feature Requests: If this is a frequent need, consider submitting a feature request to the DataHub project to support more advanced lineage path exploration features.

For more detailed instructions or examples, you might want to explore the <https://datahubproject.io/docs/generated/lineage/lineage-feature-guide|DataHub Lineage Documentation> and the <Queries | DataHub API documentation>.

Would you like more detailed instructions on using the GraphQL API or any other specific part of this process?

Sources:

3

Hi <@U08A5U4NFR7>, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.