Troubleshooting Great Expectations Integration with Datahub - Grayed Out Validation Table

ingestion
1

Original Slack Thread

Hello, I am having an issue when trying to run Great Expectations to integrate with Datahub. There is no issue sent to Datahub, but the validation table is still grayed out.attachment

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses! We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66|> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a :+1: or :-1: so other folks in the Community know if you’re still looking for help on the topic and to help us improve the assistant!

3

To better assist you with your issue, could you please provide the following information:

  1. Are you using the UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g., 0.12.0)?
  3. What data source(s) are you integrating with DataHub (e.g., BigQuery)?

This information will help in diagnosing the problem more effectively.

Additionally, based on the provided data sources, here are some common troubleshooting steps and considerations:

  1. Ensure Correct DataHub Version and URL: Make sure you are using the correct DataHub version and that the server_url in your checkpoint configuration is correctly set. For example, some users have resolved issues by changing the URL from http://<server url>:9002 to http://<server url>:8080 (https://forum.datahubproject.io/t/integrating-great-expectations-with-datahub-issue-resolved-by-changing-url-in-checkpoint-file/113|source).

  2. Check Execution Engine Compatibility: DataHub supports v3 API datasources using the SqlAlchemyExecutionEngine. If you are using a different execution engine (e.g., Spark), this might be the cause of the issue (https://datahubproject.io/docs/0.13.1/metadata-ingestion/integration_docs/great-expectations|source).

  3. Verify Dataset URNs: Ensure that the dataset URNs constructed by Great Expectations match what you have in DataHub. Any mismatch can cause the validation results not to appear (https://forum.datahubproject.io/t/troubleshooting-great-expectations-datahub-integration-and-dependency-versions/738|source).

Would you like more detailed steps or further assistance on any of these points?

Sources:

4

I check the GMS log

authenticate inbound request: Authorization header is missing 'Basic' prefix.

below is my yaml. what should i change?

    action:
      module_name: datahub.integrations.great_expectations.action
      class_name: DataHubValidationAction
      server_url: <http://localhost:8080>
      env: NON_PROD
      platform_instance_map: { "postgres": "postgres" }
      token: "Basic eyxxxx'```
5

SOLVED.

need to add additional headers