Troubleshooting Missing Tables After Ingestion

ingestion
1

Original Slack Thread

<@U06TM7M40RK> I’m not seeing my tables after ingestion

2

If you’re not seeing your tables after ingestion, there could be several potential causes and solutions to consider:

  1. Elasticsearch Limitations: If your ingestion exceeds Elasticsearch’s default limit of 10,000 items per query, some items might not be indexed correctly. You may need to adjust the page_size parameter in your ingestion recipe to ensure all items are indexed correctly. Check your Elasticsearch cluster health to ensure it can handle the volume of data being ingested (https://datahubspace.slack.com/archives/CUMUWQU66/p1726058025.657419|source).

  2. Configuration Issues: Ensure that your ingestion recipe is correctly configured. Specific configurations like projects or page_size can affect the ingestion process. Verify your ingestion recipe to ensure it is set up correctly (https://datahubspace.slack.com/archives/CUMUWQU66/p1726058025.657419|source).

  3. Permissions Problems: Ensure that the user or token used for ingestion has the necessary permissions. For example, in Tableau, the user or token must have Site Administrator Explorer permissions (https://datahubspace.slack.com/archives/CUMUWQU66/p1726058025.657419|source).

  4. DataHub UI and Backend Sync: There might be a delay in syncing the ingested data with the DataHub UI. Check the logs of your DataHub backend services (GMS, Elasticsearch, etc.) to ensure there are no errors. Restarting the services might help in syncing the data (https://datahubspace.slack.com/archives/CUMUWQU66/p1726058025.657419|source).

  5. Graph Index Out of Sync: If the graph index is out of sync, you might not see the tables in the UI. Check if Elasticsearch and Kafka are running fine (https://datahubspace.slack.com/archives/CV2UVAPPG/p1675141693.319799|source).

  6. Ingestion Logs: Check the ingestion logs for any errors or warnings that might indicate why the tables are not appearing. The logs can provide insights into any issues during the ingestion process (https://datahubproject.io/docs/0.13.1/ui-ingestion#debugging-a-failed-ingestion-run|source).

Would you like more detailed steps on any of these solutions?

Sources:

3

Hi <@U07JN5N3812>, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.

4

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><