Applying SQL Profiling with Partition Column Specification in Trino

ingestion
1

Original Slack Thread

hi, team

I’m trying to apply ‘sql profiling’ using trino, is there any way to specify ‘partition column’?
Because when I just apply ‘sql profiling’, I get an error like below.
(partition column is p_dt)

“Profiling exception (trino.exceptions.TrinoUserError) TrinoUserError(type=USER_ERROR, name=QUERY_REJECTED, message="Filter required on {database_name}.{table_name} for at least one partition column: p_dt ", query_id=20231025_010440_01174_fi4pd)\n[SQL: SELECT count(*) AS count_1 \nFROM {database_name}.{table_name}]\n(Background on this error at: https://sqlalche.me/e/14/f405)”

2

Sounds like a bug in our profiler - both some sources (e.g. bigquery) we automatically only profile the latest partition, but we haven’t rolled that out to trino yet.

If you’re up for it, we’d definitely accept a PR around this :slightly_smiling_face:

3

<@U01GZEETMEZ> I’ll try to do that if I can