Troubleshooting a Stuck Ingestion Process in BigQuery

ingestion
1

Original Slack Thread

<@U06TM7M40RK> when I trying to create ingestion source for bigquery it keeps running and getting stuck at below

[2025-04-22 06:42:47,804] INFO     {datahub.ingestion.run.pipeline:295} - Source configured successfully.
[2025-04-22 06:42:47,849] INFO     {datahub.cli.ingest_cli:130} - Starting metadata ingestion
[2025-04-22 06:42:47,850] INFO     {datahub.ingestion.source.bigquery_v2.bigquery_schema:664} - Getting projects


WARNING: These logs appear to be stale. No new logs have been received since 2025-04-22 06:42:47.850409 (271 seconds ago). However, the ingestion process still appears to be running and may complete normally.```
2

It seems likely that your ingestion process is encountering an issue where it appears to be running indefinitely without making progress. This can happen for several reasons, and there are a few steps you can take to troubleshoot and potentially resolve the issue:

  1. Check Resource Allocation: It might be a resource allocation problem. Ensure that your deployment has sufficient memory and CPU resources allocated, especially for the GMS and Elasticsearch pods, as suggested in <Slack Slack discussion>.

  2. Inspect Logs for Errors: Review the logs for any errors or warnings that might provide more context about what is causing the ingestion to hang. You can view the live logs of the pending ingestion to get more clues, as mentioned in <Slack Slack message>.

  3. Network Connectivity: Ensure that the process running the ingestion has network connectivity to the BigQuery source. The actions container must be able to resolve and connect to the BigQuery system for the ingestion command to run successfully, as noted in the <https://github.com/datahub-project/datahub/blob/master/docs/actions/actions/executor.md|Ingestion Executor documentation>.

  4. Check for Configuration Issues: Verify that your ingestion recipe is correctly configured. Misconfigurations can lead to ingestion failures. You can refer to the <https://datahubproject.io/docs/0.14.0/generated/ingestion/sources/bigquery|BigQuery ingestion documentation> for guidance on setting up your configuration.

  5. Cancel and Retry: If the ingestion run is hanging, you can cancel it and try running it again. Sometimes, restarting the process can resolve transient issues.

Would you like more detailed guidance on any of these steps?

Sources:

3

<@U07FV3UHE4D> Can I have someone from the community help me in this as I have tried all above steps given by LLM