Fixing Serialization Error in DataHub Helm Chart: Enabling Schema Registry & Upcoming Chart Version

troubleshoot
1

Original Slack Thread

Hello! I’m following <https://datahubproject.io/docs/deploy/kubernetes|Deploying with Kubernetes | DataHub> and run helm install datahub datahub/datahub in my minikube
but getting the following error….

2023-08-09 00:09:49,248 [main] ERROR c.l.d.u.s.e.steps.DataHubStartupStep:40 - DataHubStartupStep failed.
org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
Caused by: java.io.IOException: No schema registered under subject!
....```
I’m using the default value but do I need to <https://github.com/acryldata/datahub-helm/blob/bd670a922fb13f66446d26925e96c3424e546ec6/charts/prerequisites/values.yaml#L96|enable schema registry here> ? Thank you!
2

yes, there is currently a bug in the latest version. See https://github.com/acryldata/datahub-helm/issues/347 for reference

3

Thank you for your quick reply! created a PR to fix this :bow:
• <Sign in to GitHub · GitHub enable SPRING_KAFKA_PROPERTIES_AUTO_REGISTER_SCHEMAS by LittleWat · Pull Request #358 · acryldata/datahub-helm>

4

Thanks for creating the report! <@U03MF8MU5P0> Could you look into this?

5

Updated both the issue and PR with comments. For the issue, can you confirm the values settings, I am guessing KAFKA schema registry was enabled & it was pointing to GMS instead of the confluent registry service?

6

Thank you for your reply!
This datahub-0.2.181 value file.
https://github.com/acryldata/datahub-helm/blob/datahub-0.2.181/charts/datahub/values.yaml
It uses INTERNAL one. You should be able to reproduce the error.

7

just installing datahub-0.2.181 will reproduce the error.

8

I’ve had time to look at this in more detail. I see what you mean. The env var is indeed no longer needed to be set to false and a hold over from a previous method of mocking the schema registry for the upgrade job. Sorry for the delay.

9

The next version of the chart is building, 0.2.183 , thank you! :pray:

10

Thank you for your investigation and merging the PR :bow: