Troubleshooting DataHub Docker Ingestion Error

Original Slack Thread

Hello! We are exploring datahub with docker quickstart. While we were able to get the frontend and gms to work, when we started to ingest sample data, we are seeing the following error:

Downloading sample data...
Starting ingestion...
[2024-root@ip-172-30-6-163:~/.datahub/quickstart# datahub docker ingest-sample-data
Downloading sample data...
Starting ingestion...
[2024-05-03 15:51:50,203] ERROR    {datahub.entrypoints:205} - Command failed: Failed to connect to DataHub: Unable to connect to <http://localhost:8080/config> with status_code: 503.
Please check your configuration and make sure you are talking to the DataHub GMS (usually <datahub-gms-host>:8080) or Frontend GMS API (usually <frontend>:9002/api/gms).
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/run/pipeline.py", line 122, in _add_init_error_context
    yield
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/run/pipeline.py", line 218, in __init__
    self.graph.test_connection()
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/graph/client.py", line 148, in test_connection
    super().test_connection()
  File "/usr/local/lib/python3.10/dist-packages/datahub/emitter/rest_emitter.py", line 185, in test_connection
    raise ConfigurationError(message)
datahub.configuration.common.ConfigurationError: Unable to connect to <http://localhost:8080/config> with status_code: 503.
Please check your configuration and make sure you are talking to the DataHub GMS (usually <datahub-gms-host>:8080) or Frontend GMS API (usually <frontend>:9002/api/gms).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/datahub/entrypoints.py", line 192, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/datahub/telemetry/telemetry.py", line 454, in wrapper
    raise e
  File "/usr/local/lib/python3.10/dist-packages/datahub/telemetry/telemetry.py", line 403, in wrapper
    res = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/datahub/cli/docker_cli.py", line 1006, in ingest_sample_data
    pipeline = Pipeline.create(recipe, no_default_report=True)
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/run/pipeline.py", line 365, in create
    return cls(
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/run/pipeline.py", line 215, in __init__
    with _add_init_error_context("connect to DataHub"):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/datahub/ingestion/run/pipeline.py", line 124, in _add_init_error_context
    raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to connect to DataHub: Unable to connect to <http://localhost:8080/config> with status_code: 503.
Please check your configuration and make sure you are talking to the DataHub GMS (usually <datahub-gms-host>:8080) or Frontend GMS API (usually <frontend>:9002/api/gms).```

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!

Datahub version: v0.13.2
Also, our goal is to configure a push based ingestion from our Databricks cluster. For that, we are not sure where we need to add/generate the DATAHUB_REST_TOKEN in the docker compose file. Can someone please help?