Error with Looker Module in DataHub Intake

ingestion
1

Original Slack Thread

Good afternoon, I have a problem with an intake, with the looker module, this is the error:

<html>
<head>
    <title>Looker Not Found (404)</title>
    &lt;link href='<https://fonts.googleapis.com/css?family=Open+Sans:400,600>' rel='stylesheet' type='text/css'&gt;

    &lt;!-- @@@@@@@@@@@@@ FAVICONS @@@@@@@@@@@@@ --&gt;
    &lt;style type="text/css"&gt;
    &lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;

    &lt;div class="message"&gt;

        &lt;img width="210" height="84" src="<https://wwwstatic-a.lookercdn.com/logos/looker_all_white.svg>" alt="Looker"&gt;

        &lt;h1&gt;Looker is unavailable.&lt;/h1&gt;

        &lt;h2&gt;If you typed in a URL, double-check the spelling.&lt;/h2&gt;
        &lt;h2&gt;This may also be due to a temporary condition such as an outage, &lt;a href="<https://docs.looker.com/relnotes/hosted-maintenance-hours>"&gt;scheduled maintenance&lt;/a&gt; or upgrade.&lt;/h2&gt;
        &lt;br&gt;
        &lt;h3&gt;
          If this message persists or you have any concerns, &lt;br&gt; contact us from
          &lt;a href="<https://help.looker.com>"&gt;<http://help.looker.com|help.looker.com>&lt;/a&gt; and we'll respond promptly.
        &lt;/h3&gt;

    &lt;/div&gt;

&lt;/body&gt;
&lt;/html&gt;
Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/rtl/serialize.py", line 66, in deserialize
    data = json.loads(data)
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/rtl/api_methods.py", line 90, in _return
    sdk_error = self.deserialize(data=value, structure=error.SDKError) # type: ignore
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/rtl/serialize.py", line 68, in deserialize
    raise DeserializeError(f"Bad json {ex}")
looker_sdk.rtl.serialize.DeserializeError: Bad json Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/entrypoints.py", line 188, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 454, in wrapper
    raise e
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 403, in wrapper
    res = func(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 201, in run
    ret = loop.run_until_complete(run_ingestion_and_check_upgrade())
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 185, in run_ingestion_and_check_upgrade
    ret = await ingestion_future
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 139, in run_pipeline_to_completion
    raise e
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 131, in run_pipeline_to_completion
    pipeline.run()
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 404, in run
    for wu in itertools.islice(
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 126, in auto_stale_entity_removal
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 150, in auto_workunit_reporter
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 206, in re_emit_browse_path_v2
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 247, in auto_browse_path_v2
    for urn, batch in _batch_workunits_by_urn(stream):
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 385, in _batch_workunits_by_urn
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 163, in auto_materialize_referenced_tags
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 70, in auto_status_aspect
    for wu in stream:
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/source/looker/looker_source.py", line 1316, in get_workunits_internal
    yield from self.extract_independent_looks()
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/source/looker/looker_source.py", line 1202, in extract_independent_looks
    query: Query = self.looker_api.get_query(look.query_id, query_fields)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/ingestion/source/looker/looker_lib_wrapper.py", line 252, in get_query
    return self.client.query(
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/sdk/api40/methods.py", line 8801, in query
    self.get(
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/rtl/api_methods.py", line 149, in get
    return self._return(response, structure)
  File "/tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/looker_sdk/rtl/api_methods.py", line 92, in _return
    raise error.SDKError(value)
looker_sdk.error.SDKError: <!DOCTYPE html>
&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Looker Not Found (404)&lt;/title&gt;
    &lt;link href='<https://fonts.googleapis.com/css?family=Open+Sans:400,600>' rel='stylesheet' type='text/css'&gt;

    &lt;!-- @@@@@@@@@@@@@ FAVICONS @@@@@@@@@@@@@ --&gt;

    &lt;style type="text/css"&gt;
        body {
            background-color: #2e343f;
            color: white;
            height: auto;
            font-family: Open Sans, Helvetica, Arial, sans-serif;
        }
        .message {
            width: 100%;
            max-width: 760px;
            margin: 0 auto;
            margin-top: 135px;
            text-align: center;
        }
        h2, h3 {
            font-weight: normal;
        }
        a {
            color: white;
        }
    &lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;

    &lt;div class="message"&gt;

        &lt;img width="210" height="84" src="<https://wwwstatic-a.lookercdn.com/logos/looker_all_white.svg>" alt="Looker"&gt;

        &lt;h1&gt;Looker is unavailable.&lt;/h1&gt;

        &lt;h2&gt;If you typed in a URL, double-check the spelling.&lt;/h2&gt;
        &lt;h2&gt;This may also be due to a temporary condition such as an outage, &lt;a href="<https://docs.looker.com/relnotes/hosted-maintenance-hours>"&gt;scheduled maintenance&lt;/a&gt; or upgrade.&lt;/h2&gt;
        &lt;br&gt;
        &lt;h3&gt;
          If this message persists or you have any concerns, &lt;br&gt; contact us from
          &lt;a href="<https://help.looker.com>"&gt;<http://help.looker.com|help.looker.com>&lt;/a&gt; and we'll respond promptly.
        &lt;/h3&gt;

    &lt;/div&gt;

&lt;/body&gt;
&lt;/html&gt;

[2024-06-27 19:32:18,590] DEBUG    {datahub.entrypoints:203} - DataHub CLI version: 0.12.1.5 at /tmp/datahub/ingest/venv-looker-0.12.1.5/lib/python3.10/site-packages/datahub/__init__.py
[2024-06-27 19:32:18,590] DEBUG    {datahub.entrypoints:206} - Python version: 3.10.11 (main, May 23 2023, 13:58:30) [GCC 10.2.1 20210110] at /tmp/datahub/ingest/venv-looker-0.12.1.5/bin/python3 on Linux-6.1.85+-x86_64-with-glibc2.31
[2024-06-27 19:32:18,591] DEBUG    {datahub.entrypoints:211} - GMS config {'models': {}, 'patchCapable': True, 'versions': {'linkedin/datahub': {'version': 'v0.12.1', 'commit': '159a013b0515f8a94b88d62e4ad20aad228fac9d'}}, 'managedIngestion': {'defaultCliVersion': '0.12.1.1', 'enabled': True}, 'statefulIngestionCapable': True, 'supportsImpactAnalysis': True, 'timeZone': 'GMT', 'telemetry': {'enabledCli': True, 'enabledIngestion': False}, 'datasetUrnNameCasing': False, 'retention': 'true', 'datahub': {'serverType': 'prod'}, 'noCode': 'true'}```
Has anyone had this error happen?
2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Are you using UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g. 0.12.0)
  3. What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a :thumbsup: or :thumbsdown: so other folks in the Community know if you’re still looking for help on the topic!

3

Good afternoon! To assist you better, could you provide the following details?

  1. Are you using the UI or CLI for ingestion?
  2. Which DataHub version are you using? (e.g., 0.12.0)?
  3. What data source(s) are you integrating with DataHub (e.g., Looker, BigQuery)?

Based on the error message, it seems like the Looker API endpoint you are trying to reach is unavailable, possibly due to a misconfiguration or an outage. Here are a few steps to troubleshoot this issue:

  1. Check Looker URL: Ensure that the Looker base URL you have configured is correct and accessible. You can verify this by trying to access the URL directly in a web browser.
  2. API Version: Make sure you are using the correct API version. According to <Slack source>, DataHub supports Looker API v4. Ensure your API credentials are compatible with this version.
  3. Looker Availability: Verify that Looker is not undergoing maintenance or experiencing an outage. You can check Looker’s status page or contact Looker support for more information.

Would you like more detailed steps on how to verify these configurations?

Sources:

  • <Slack Slack Thread on Looker API Issues> :+1: 1 button :-1: 0 button :bulb: Hint: Mention <@U06TM7M40RK> in the thread for followups.
4

i use UI for ingestion, the datahub version is 0.12.1, the data source is Looker <@U06TM7M40RK>