Reading CSI Mounted Secret from Ingestion Recipe in DataHub

ingestion
1

Original Slack Thread

<@U06TM7M40RK> how to read csi mounted secret from ingestion recipe

2

To read a CSI (Container Storage Interface) mounted secret from an ingestion recipe in DataHub, you can follow these steps:

  1. Mount the Secret in Kubernetes: First, ensure that your secret is mounted as a volume in your Kubernetes pod. This is typically done using a SecretProviderClass with the CSI driver. For example, you might have a volume mount configuration like this:

    extraVolumeMounts:
  - mountPath: /mnt/secrets-store
    name: secrets-store-inline
    readOnly: true
extraVolumes:
  - name: secrets-store-inline
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: your-secret-provider-class

  2. Access the Secret in the Recipe: In your ingestion recipe, you can reference the secret by specifying the path where the secret is mounted. For example, if your secret is a password stored in a file at /mnt/secrets-store/mysql-password, you can reference it directly in your recipe:

    source:
  type: mysql
  config:
    host_port: 'localhost:3306'
    database: my_db
    username: root
    password: /mnt/secrets-store/mysql-password

  3. Ensure the File is Readable: Make sure that the file permissions allow the DataHub process to read the secret file. This typically means ensuring that the file is readable by the user under which the DataHub process is running.

  4. Test the Configuration: After setting up the volume mount and updating your recipe, test the ingestion process to ensure that the secret is being read correctly.

If you need more detailed steps or have specific questions about this process, feel free to ask!

Sources:

  • <Slack Discussion on CSI Mounted Secrets> :+1: 0 button :-1: 0 button :light_bulb: Hint: Mention @runllm in the thread for followups.
3

Hi <@U08PVDCPJAG>, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.