Internal Schema Registry only really works on fresh installs because it has hardcoded schema ids it registers, so the existing mappings on the broker/zookeeper cause issues. If you’re able to wipe those volumes without risking data then it should work. The most well supported deployment is just with Confluent’s schema registry deployed. We don’t really heavily utilize the schema registry since our topic schemas are all designed as GenericRecords, which is part of why we made steps to move towards internal.
Ok thanks for the information. In that case I will not switch to using “INTERNAL”. Any idea on how I could solve this with still using AWS Glue as schema registry? Is this something that will be fixed on the next release? Do you want me to report a bug on Github?
Yeah go ahead and file a bug on Github, will try to get this fixed by the next release.
Here is the issue: https://github.com/datahub-project/datahub/issues/8726