Hello Folks,
we have tried to create lineage for postgres DB but after ingestion I have to manually create the lineage between the table. May i know what could be the reason. Can I get the lineage view automatically.
To automatically generate lineage for a PostgreSQL database in DataHub, you need to ensure that your ingestion configuration is set up correctly. Here are some steps and considerations based on the available data sources:
Configuration Settings: Make sure that your ingestion recipe includes the necessary configurations for capturing lineage. Specifically, you need to enable include_view_lineage if you want to capture lineage from views. Here’s an example configuration:
According to the <Postgres | DataHub Ingestion Documentation>, these settings are crucial for enabling lineage capture.
Check for Errors: Review the logs for any errors or warnings that might indicate why lineage is not being captured. Logs can provide insights into configuration issues or other problems during ingestion.
Version Compatibility: Ensure that the DataHub version you are using supports the features you need. Some features might not be available in older versions. As per the <https://datahubproject.io/docs/0.14.0/generated/lineage/lineage-feature-guide|DataHub Lineage Guide>, automatic lineage extraction is supported for various data tools, including PostgreSQL.
Schema and Table Patterns: Verify that your schema and table patterns are correctly configured to include the tables and views for which you want to capture lineage.
If you have followed these steps and are still experiencing issues, it might be helpful to provide specific logs or error messages for further troubleshooting. Would you like more detailed steps or have specific logs/errors to share? This will help in providing a more targeted solution.
Hi, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.
Thank you for letting us know you need a person to take a look at this thread.
Either a community member or one of our team members will try to assist when they have a moment, responses unfortunately might not be immediate. We hope this helps speed things along