Hi Team!
I have a problem with SSL certificates Apache Superset. Can u help me? Maybe i must change my ingestion recipe?
Error log
Execution finished with errors.
{'exec_id': '9c78f121-03f4-4693-b30b-1bc1a283eb1a',
'infos': ['2024-06-05 07:58:49.669538 INFO: Starting execution for task with name=RUN_INGEST',
"2024-06-05 07:58:53.715320 INFO: Failed to execute 'datahub ingest', exit code 1",
'2024-06-05 07:58:53.715496 INFO: Caught exception EXECUTING task_id=9c78f121-03f4-4693-b30b-1bc1a283eb1a, name=RUN_INGEST, '
'stacktrace=Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 140, in execute_task\n'
' task_event_loop.run_until_complete(task_future)\n'
' File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
' return future.result()\n'
' File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 282, in execute\n'
' raise TaskError("Failed to execute \'datahub ingest\'")\n'
"acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
'errors': []}
~~~~ Ingestion Logs ~~~~
Obtaining venv creation lock...
Acquired venv creation lock
venv is already set up
venv setup time = 0 sec
This version of datahub supports report-to functionality
+ exec datahub ingest run -c /tmp/datahub/ingest/9c78f121-03f4-4693-b30b-1bc1a283eb1a/recipe.yml --report-to /tmp/datahub/ingest/9c78f121-03f4-4693-b30b-1bc1a283eb1a/ingestion_report.json
[2024-06-05 07:58:52,376] INFO {datahub.cli.ingest_cli:147} - DataHub CLI version: 0.13.3rc1
[2024-06-05 07:58:52,419] INFO {datahub.ingestion.run.pipeline:254} - Sink configured successfully.
[2024-06-05 07:58:52,751] ERROR {datahub.entrypoints:205} - Command failed: Failed to configure the source (superset): HTTPSConnectionPool(host='<http://superset.dev.datastorage.int.rolfcorp.ru|superset.dev.datastorage.int.rolfcorp.ru>', port=443): Max retries exceeded with url: /login/api/v1/security/login (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
conn.connect()
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connection.py", line 653, in connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connection.py", line 806, in _ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 465, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 509, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/lib/python3.10/ssl.py", line 1104, in _create
self.do_handshake()
File "/usr/local/lib/python3.10/ssl.py", line 1375, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
raise new_e
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<http://superset.dev.datastorage.int.rolfcorp.ru|superset.dev.datastorage.int.rolfcorp.ru>', port=443): Max retries exceeded with url: /login/api/v1/security/login (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 123, in _add_init_error_context
yield
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 272, in __init__
self.source = source_class.create(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/superset.py", line 219, in create
return cls(ctx, config)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/superset.py", line 182, in __init__
login_response = <http://requests.post|requests.post>(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/requests/adapters.py", line 698, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='<http://superset.dev.datastorage.int.rolfcorp.ru|superset.dev.datastorage.int.rolfcorp.ru>', port=443): Max retries exceeded with url: /login/api/v1/security/login (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/entrypoints.py", line 192, in main
sys.exit(datahub(standalone_mode=False, **kwargs))
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 454, in wrapper
raise e
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 403, in wrapper
res = func(*args, **kwargs)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 201, in run
ret = loop.run_until_complete(run_ingestion_and_check_upgrade())
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 170, in run_ingestion_and_check_upgrade
pipeline = Pipeline.create(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 388, in create
return cls(
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 271, in __init__
with _add_init_error_context(f"configure the source ({self.source_type})"):
File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/tmp/datahub/ingest/venv-superset-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 125, in _add_init_error_context
raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to configure the source (superset): HTTPSConnectionPool(host='<http://superset.dev.datastorage.int.rolfcorp.ru|superset.dev.datastorage.int.rolfcorp.ru>', port=443): Max retries exceeded with url: /login/api/v1/security/login (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))```
my recipe
```run_id: 'urn:li:dataHubExecutionRequest:9c78f121-03f4-4693-b30b-1bc1a283eb1a'
sink:
type: datahub-kafka
config:
connection:
bootstrap: 'broker:29092'
schema_registry_url: '<http://schema-registry:8081>'
source:
type: superset
config:
connect_uri: '<https://superset.dev.datastorage.int.rolfcorp.ru/login/>'
password: '${DWH_SUPERSET_PASSWORD}'
provider: db
username: admin
pipeline_name: 'urn:li:dataHubIngestionSource:1d9947ae-f280-47d3-b553-3fb94ff22e2a'```