Troubleshooting Elasticsearch Ingestion Timing Out

Original Slack Thread

This message was deleted.

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

The issue you’re experiencing with the Elasticsearch ingestion timing out could be due to several factors. Based on the error message and the information from the data sources, here are some potential causes and solutions:

  1. Network Connectivity: According to the <Ingestion | DataHub documentation>, network reachability issues can cause ingestion failures. Ensure that the network where DataHub is deployed has access to the Elasticsearch cluster. This includes checking firewall settings and ensuring that the Elasticsearch host is reachable from the DataHub environment.

  2. Firewall Settings: As suggested in a <Slack discussion>, check the firewall settings on the Elasticsearch host to ensure that it is not blocking incoming connections from the DataHub server.

  3. Resource Allocation: If the Elasticsearch or broker container is not allocated enough resources, it might lead to connection issues. The <Quickstart Debugging Guide | DataHub guide> recommends allocating at least 8GB of RAM and 2GB of swap space to Docker.

  4. Timeout Configuration: The default connection timeout might be too short for your network conditions. Consider increasing the timeout settings in your configuration to allow more time for the connection to be established.

Would you like more detailed steps on how to check and modify these settings?

Sources:

<@U06TM7M40RK> how do you check whitelisting for ip

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><