Update Model Test Validation Logic from Manifest JSON to Run Results JSON File

ingestion
1

Original Slack Thread

view logic query of model test validation is getting updated using manifest json instead of run results json file

2

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

It seems likely that you’re experiencing an issue where the logic for model test validation is being updated using the manifest.json file instead of the run_results.json file. This could be due to the way the DataHub ingestion process is configured or executed.

To address this, it might be helpful to ensure that the run_results.json file is correctly integrated into your DataHub ingestion process. According to the <dbt | DataHub documentation>, the run_results.json file is crucial for capturing the results of dbt test runs and should be included in the ingestion recipe. Here are some steps you might consider:

  1. Ensure Correct File Paths: Verify that the path to the run_results.json file is correctly specified in your ingestion configuration. This file should be accessible to the DataHub ingestion process.

  2. Copy and Backup: As per the https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/docs/sources/dbt/dbt.md|documentation, it’s recommended to copy the run_results.json file to a backup location before running other dbt commands, as subsequent commands might overwrite it.

  3. Ingestion Recipe Configuration: Make sure that your ingestion recipe includes the run_results_paths configuration pointing to the correct location of the run_results.json file. This is necessary for the ingestion process to utilize the test results.

  4. Check for Overwrites: If the run_results.json file is being overwritten, ensure that you have a backup strategy in place to preserve the test results for ingestion.

Would you like more detailed guidance on any of these steps or further assistance with your specific setup?

Sources:

4

<@U07FV3UHE4D> please help for the above issue,
view logic is getting picked from manifest which is having raw code , I want to publish correct logic query like below which is available in run results json![attachment]({‘ID’: ‘F086RFLBYKT’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U07EV85700N’, ‘CREATED’: ‘2024-12-24 09:25:10+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-12-24 09:25:10+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F086RFLBYKT-7d4d026bdc’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1735032315.579489’, ‘PARENT_MESSAGE_TS’: ‘1734973613.257489’, ‘MESSAGE_CHANNEL_ID’: ‘CUMUWQU66’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 661235, ‘_FIVETRAN_SYNCED’: ‘2024-12-29 08:22:42.768000+00:00’})

5

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><