Original Slack Thread
Hello team,
My question related to great expectations
just wanted to know whether GX supports custom query (where clause)? Example: select * from table_name where organigation = ‘apple’ and created_date = ‘2025-02-05’
I want to validate the assertions with the above query
Cc <@U054NUV9D9T>
Great Expectations (GX) itself does support filtering data using custom queries or “batch_kwargs” that include a SQL query or a WHERE clause, depending on the execution engine and how you configure your expectation suite. However, your question is about using such custom queries in the context of DataHub’s integration with Great Expectations.
DataHub’s GX Integration:
Limitations:
Summary:
Yes, you can use custom queries (with WHERE clauses) in Great Expectations when using SQLAlchemyExecutionEngine, and DataHub will ingest the results if you configure the integration as described in the <https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/integration_docs/great-expectations.md|official documentation>. For other engines or more advanced use cases, this is not currently supported out of the box.
Would you like more detail on how to set up such a custom query in your GX configuration for DataHub?
Sources: