Hi Everyone, I’m running Datahub v0.11 and have a working Datahub install with EKS and NodeGroups (Ec2).
I’m trying to move the entire cluster to EKS Fargate (serverless). Kafka and Zookeeper need persistent storage, and I’ve set up and EFS file system for those two pods.
The pre-requisites pods start and stay running, but Kafka crashes from time to time. When I run the datahub helms, I keep seeing this error in the logs and the kafka setup job keeps failing. Does anyone have any troubleshooting ideas?
[2024-02-21 22:46:41,386] WARN [LegacyAdminClient clientId=admin-1] Bootstrap broker prerequisites-kafka:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
Exception in thread "main" java.lang.RuntimeException: Request METADATA failed on brokers List(prerequisites-kafka:9092 (id: -1 rack: null))at kafka.admin.BrokerApiVersionsCommand$AdminClient.sendAnyNode(BrokerApiVersionsCommand.scala:156)
at kafka.admin.BrokerApiVersionsCommand$AdminClient.findAllBrokers(BrokerApiVersionsCommand.scala:179)
at kafka.admin.BrokerApiVersionsCommand$AdminClient.awaitBrokers(BrokerApiVersionsCommand.scala:171)
at kafka.admin.BrokerApiVersionsCommand$.execute(BrokerApiVersionsCommand.scala:61)
at kafka.admin.BrokerApiVersionsCommand$.main(BrokerApiVersionsCommand.scala:55)
at kafka.admin.BrokerApiVersionsCommand.main(BrokerApiVersionsCommand.scala)