Issue with Tableau Ingestion KeyError: '272322c6-0191-42da-90bd-04856b3bed5b'

user-2 · July 15, 2024, 12:05am

Unable to ingest Tableau

user-2 · July 15, 2024, 12:05am

[2024-07-12 10:29:12,124] INFO {datahub.ingestion.reporting.file_reporter:54} - Wrote UNKNOWN report successfully to <_io.TextIOWrapper name=‘/tmp/datahub/ingest/4555cc7d-6205-4645-9d42-022a4fa969b8/ingestion_report.json’ mode=‘w’ encoding=‘UTF-8’>
[2024-07-12 10:29:12,125] INFO {datahub.cli.ingest_cli:133} - Source (tableau) report:
{‘events_produced’: 0,
‘events_produced_per_sec’: 0,
‘entities’: {},
‘aspects’: {},
‘aspect_urn_samples’: {},
‘warnings’: {},
‘failures’: {},
‘soft_deleted_stale_entities’: ,
‘start_time’: ‘2024-07-12 10:29:03.632583 (8.49 seconds ago)’,
‘running_time’: ‘8.49 seconds’}
[2024-07-12 10:29:12,126] INFO {datahub.cli.ingest_cli:136} - Sink (datahub-rest) report:
{‘total_records_written’: 0,
‘records_written_per_second’: 0,
‘warnings’: ,
‘failures’: ,
‘start_time’: ‘2024-07-12 10:29:03.497124 (8.63 seconds ago)’,
‘current_time’: ‘2024-07-12 10:29:12.126143 (now)’,
‘total_duration_in_seconds’: 8.63,
‘max_threads’: 15,
‘gms_version’: ‘v0.13.3rc1’,
‘pending_requests’: 0}
[2024-07-12 10:29:12,551] ERROR {datahub.entrypoints:205} - Command failed: ‘272322c6-0191-42da-90bd-04856b3bed5b’
Traceback (most recent call last):
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/entrypoints.py”, line 192, in main
sys.exit(datahub(standalone_mode=False, **kwargs))
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/click/core.py”, line 783, in invoke
return __callback(*args, **kwargs)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/telemetry/telemetry.py”, line 454, in wrapper
raise e
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/telemetry/telemetry.py”, line 403, in wrapper
res = func(*args, **kwargs)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py”, line 201, in run
ret = loop.run_until_complete(run_ingestion_and_check_upgrade())
File “/usr/local/lib/python3.10/asyncio/base_events.py”, line 649, in run_until_complete
return future.result()
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py”, line 185, in run_ingestion_and_check_upgrade
ret = await ingestion_future
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py”, line 139, in run_pipeline_to_completion
raise e
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/cli/ingest_cli.py”, line 131, in run_pipeline_to_completion
pipeline.run()
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py”, line 430, in run
for wu in itertools.islice(
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 162, in auto_stale_entity_removal
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 186, in auto_workunit_reporter
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 279, in auto_browse_path_v2
for urn, batch in _batch_workunits_by_urn(stream):
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 494, in _batch_workunits_by_urn
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 394, in auto_fix_duplicate_schema_field_paths
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 203, in auto_materialize_referenced_tags_terms
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py”, line 106, in auto_status_aspect
for wu in stream:
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/tableau.py”, line 2801, in get_workunits_internal
self._populate_projects_registry()
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/tableau.py”, line 775, in _populate_projects_registry
all_project_map: Dict[str, TableauProject] = self._get_all_project()
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/tableau.py”, line 678, in _get_all_project
set_project_path()
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/tableau.py”, line 673, in set_project_path
project.path = form_path(project_id=project.id)
File “/tmp/datahub/ingest/venv-tableau-03575587e416950c/lib/python3.10/site-packages/datahub/ingestion/source/tableau.py”, line 668, in form_path
cur_proj = all_project_map[cur_proj.parent_id]
KeyError: ‘272322c6-0191-42da-90bd-04856b3bed5b’

datahub_team · July 15, 2024, 12:05am

Hey there! Make sure your message includes the following information if relevant, so we can help more effectively!

Are you using UI or CLI for ingestion?
Which DataHub version are you using? (e.g. 0.12.0)
What data source(s) are you integrating with DataHub? (e.g. BigQuery)

<@U06TM7M40RK> Can you help with the questions in this thread?

We’re experimenting with a new AI assistant for questions in the <#CUMUWQU66> channel to help you track down helpful resources more quickly. Keep in mind that responses may not always be spot-on; feel free to ask it follow-up questions and don’t forget to provide its responses a or so other folks in the Community know if you’re still looking for help on the topic!

user-1 · July 15, 2024, 12:05am

To help you better, I need a bit more information:

Are you using the UI or CLI for ingestion?
Which DataHub version are you using?
What data source(s) are you integrating with DataHub? (e.g., Tableau, BigQuery)?

From the error logs you provided, it seems like there is a KeyError related to a missing project ID during the ingestion process. This might be due to an issue with the way projects are being fetched or processed.

Based on the data sources, there are a few known issues and potential solutions related to Tableau ingestion:

ProjectLuid Field Issue: If you’re using a Tableau version earlier than 2022.3, there might be issues with the projectLuid field. This was a breaking change introduced in later versions of Tableau (https://github.com/datahub-project/datahub/issues/7575|source).
Permissions Issue: Ensure that the user account used for ingestion has the necessary permissions, such as Site Administrator Explorer (https://datahubspace.slack.com/archives/C029A3M079U/p1682430052.651439|source).
Elasticsearch Limitation: If you’re hitting a limit of 10,000 objects, it might be due to Elasticsearch’s default settings (https://datahubspace.slack.com/archives/C029A3M079U/p1712327114.398919|source).
Subprojects Handling: Issues with nested subprojects can also cause ingestion failures (https://github.com/datahub-project/datahub/issues/9398|source).

Would you like more detailed steps to troubleshoot this issue, or do you have specific aspects you’d like to focus on?

Sources:

<Issues · datahub-project/datahub · GitHub change in tableau ingestion>
<Issues · datahub-project/datahub · GitHub ingestion error: Cannot return null for non-nullable type>
<Slack ingestion problem>
<Slack data ingestion error> 0 button 0 button Hint: Mention @runllm in the thread for followups.

user-2 · July 15, 2024, 12:05am

UI
Datahub version - v0.13.3rc1
Tableau

user-2 · July 15, 2024, 12:05am

Tableau Version -> 2023.3.2 (20233.24.0112.1233)

Topic		Replies	Views
Troubleshooting Tableau Data Ingestion Timeout Errors ingestion	8	5	July 22, 2024
Troubleshooting Tableau Server Data Ingestion Error: 403 Forbidden ingestion	8	51	June 17, 2024
Troubleshooting DataHub Ingestion Issues ingestion	2	46	September 2, 2024
Troubleshooting Tableau Ingestion Discrepancy with DataHub Instances troubleshoot	2	45	April 8, 2024
Troubleshooting Tableau Ingestion Errors in DataHub 0.13.1 troubleshoot	1	71	April 15, 2024

Issue with Tableau Ingestion KeyError: '272322c6-0191-42da-90bd-04856b3bed5b'

Related topics