Troubleshooting Kafka Startup Issue during Upgrade from 0.12.1 to 1.13.2

all-things-deployment
1

Original Slack Thread

Hello, upgrading from 0.12.1 to 1.13.2 and have some issues. Using datahub quickstart on a single machine. Kafka does not seem to start up. Getting error in the broker docker:
[2024-05-03 07:18:18,582] ERROR Error while creating ephemeral at /brokers/ids/1, node already exists and owner ‘72900087829037057’ does not match current session ‘72900095327076353’ (kafka.zk.KafkaZkClient$CheckedEphemeral)
[2024-05-03 07:18:18,590] ERROR [KafkaServer id=1] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)

Any help much appreciated!

datahub docker quickstart --quickstart-compose-file da-datahub-custom/docker-compose.yml
Docker-compose file: https://pastebin.com/YHDY71gf

2

you may give try deleting the broker and zookeeper volume and start the quick start

docker volume rm  datahub_broker datahub_zkdata
datahub docker quickstart --quickstart-compose-file da-datahub-custom/docker-compose.yml```
3

Thanks! That wont delete any datahub content?

4

it won’t delete any metadata that is already ingested into Datahub. Any MCP/MCE messages that are in Kafka but not yet consumed by the mce/mae consumer will be lost. In the case of Datahub Quickstart, it is unlikely that you might have anything remaining in Kafka.