Importing JSON Schemas from Apicurio Schema Registry into DataHub

Original Slack Thread

Is there an option to import json schemas form Apicurio schema registry in DataHub

Hey there! :wave:

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

To import JSON schemas from an Apicurio schema registry into DataHub, you would typically need to set up a custom ingestion process. While there is existing support for ingesting schemas from the Confluent Schema Registry, adapting this for Apicurio would require some customization.

According to a <Slack conversation>, there is code that ingests schemas from the Confluent Schema Registry as part of Kafka metadata ingestion, and it should be possible to repurpose this for Apicurio. This would likely involve creating a custom ingestion recipe or script to handle the specifics of the Apicurio schema registry.

Additionally, DataHub supports JSON schema ingestion, as noted in the <https://datahubproject.io/docs/0.14.0/generated/ingestion/sources/json-schema|DataHub documentation>. This involves using the JSON schema source to extract metadata from JSON schemas, which can be configured to point to a specific path or URL where the schemas are located.

Would you like more detailed guidance on setting up a custom ingestion process for Apicurio, or do you have any specific questions about the configuration?

Sources:

Can you help me with more guidance on how to set up ingestion process for Apicurio schema registry <@U06TM7M40RK>

Hi :slight_smile: ! Did this interaction resolve your issue or do you still require assistance ?

Please tag @ RunLLM if you still need further assistance !

Or tag @Datahub Community Support if you require a person to take a look :slight_smile:

Hope this helps ! ><