Hi everyone: There is a mandatory Looker API Change coming down.
https://cloud.google.com/looker/docs/best-practices/query-id-update-instructions
Anyone know if the DH Ingestion is using <query_id> or GET /queries/slug<slug> ?
Hey there! Make sure your message includes the following information if relevant, so we can help more effectively!
- Are you using UI or CLI for ingestion?
- Which DataHub version are you using? (e.g. 0.12.0)
- What data source(s) are you integrating with DataHub? (e.g. BigQuery)
I actually see the following in the looker_source.py module:
all_looks: List[Look] = self.looker_api.all_looks(
fields=look_fields, soft_deleted=self.source_config.include_deleted
)
for look in all_looks:
if look.id in self.reachable_look_registry:
# This look is reachable from Dashboard
continue
if look.query_id is None:
<http://logger.info|logger.info>(f”query_id is None for look {look.title}({look.id})“)
continue
query: Query = self.looker_api.get_query(look.query_id, query_fields)
But, I do see this on line 1164-1188 of looker_source.py Getting slug as the last query field.
def extract_independent_looks(self) -> Iterable[MetadataWorkUnit]:
“”"
Emit MetadataWorkUnit for looks which are not part of any Dashboard
“”"
if self.source_config.extract_independent_looks is False:
return
self.reporter.report_stage_start(“extract_independent_looks”)
logger.debug(“Extracting looks not part of Dashboard”)
look_fields: List[str] = [
“id”,
“title”,
“description”,
“query_id”,
]
query_fields: List[str] = [
“id”,
“view”,
“model”,
“dynamic_fields”,
“filters”,
“fields”,
“slug”,
]
From what I can tell, we don’t ever directly query queries (or render_tasks or query running queries) so I think we will be okay here. Thanks for the heads up!
Thanks Andrew, I am looking into this further just to confirm.