Resolving SSL Certificate Error in Python Kafka Emitter for Schema Registry

all-things-deployment
41

Curl —cacert etc/ssl/certs/ca.cry (url)
I am using this command and getting response

42

Yeah same one

43

How can we disable verification in Kafka emitter then in order to resolve this issue ?

44

I can see no option in Kafka emitter code

45

For schema_registry if we want to disable verification

46

I checked the code, and I’m confident the set properties are passed to the Schema Registry client.
Here is my test recipe:

  type: "kafka"
  config:
    connection:
      bootstrap: "localhost:9092"
      #consumer_config:
      #  security.protocol: "SASL_SSL"
      #  sasl.mechanism: "PLAIN"
      #  sasl.username: "kafka_key"
      #  sasl.password: "kafka_secret"
      schema_registry_url: "<http://localhost:8081>"
      schema_registry_config:
        # See <https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#confluent_kafka.schema_registry.SchemaRegistryClient>
        # Most of these config keys are optional.
        ssl.ca.location: "certificate_ca.pem"
        ssl.certificate.location: "certificate.pem"
        ssl.key.location: "private_key.pem"
        ssl.key.password: "XXXXXXX"```
47

based on the Confluent Schema Registry Python code there is a way to disable SSL cert verification, even though it is a bit hacky, if you set the ssl.ca.location to false

48

I am using Kafka emitter only and I am setting all these configuration like this

49

Does we need to pass these to schema registry client as it’s already a Kafka emitter

50

For the emitter the config looks like this:

  type: "datahub-kafka"
  config:
    connection:
      bootstrap: localhost:9092
      schema_registry_url: <http://localhost:8081>
      schema_registry_config:
        # See <https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#confluent_kafka.schema_registry.SchemaRegistryClient>
        # Most of these config keys are optional.
        #ssl.ca.location: "certificate_ca.pem"
        ssl.certificate.location: "certificate.pem"
        ssl.key.location: "private_key.pem"```
51

and here I verified as well the config properties are passed to the Conflunet Schema Registry client

52

I have this python code for Kafka emitter and all the configuration are passed from action pipeline

53

Like this

54

Is it correct ?

55

As this is a custom code, I would debug your code if it passes properly the parameters you set.
Unfortunately, I can only test/check what is in our codebase and there the Kafka Emitter should pass the property if it is passed properly

56

As I said another thing your can try to set the ssl.ca.location to False if you are sure you set that property and you want to disable verifying the server’s certificate

57

Let me share you

58

Can’t you debug your code in standalone mode without the action framework?

59

just to see if the Kafka Emitter gets the proper parameters

60

Yeah debug that it’s working fine