<@U06TM7M40RK> how to use secret from ingestion recipe for /mnt/secrets mounted in datahub-actions
To use secrets from a mounted directory like /mnt/secrets
in your DataHub ingestion recipe, you can follow these steps:
-
Mount the Secret: Ensure that your Kubernetes setup mounts the secret to the
/mnt/secrets
directory in thedatahub-actions
pod. This is typically done using a Kubernetes Secret or a CSI driver for secret management like AWS Secrets Manager or Azure Key Vault. -
Access the Secret in the Recipe: In your ingestion recipe, you can reference the secret by reading the file directly from the mounted path. This involves using the
__DATAHUB_TO_FILE_<property>
directive in your recipe to specify that the value should be read from a file.
Here’s an example of how you might configure your ingestion recipe to use a secret stored in a file:
source:
type: mysql
config:
host_port: "localhost:3306"
database: "my_db"
username: "root"
# Use the __DATAHUB_TO_FILE directive to read the password from a file
__DATAHUB_TO_FILE_password: "/mnt/secrets/mysql-password"
include_tables: true
include_views: true
profiling:
enabled: true
sink:
type: datahub-rest
config:
server: "http://datahub-gms:8080"
In this example, the password
is read from the file located at /mnt/secrets/mysql-password
. The __DATAHUB_TO_FILE_password
directive tells DataHub to read the password from the specified file path.
- Ensure Proper Permissions: Make sure that the
datahub-actions
pod has the necessary permissions to read from the/mnt/secrets
directory. This might involve setting the correct file permissions and ensuring that the Kubernetes service account has access to the secret.
Would you like more detailed steps or further assistance on any specific part of the process?
Sources:
- <https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/recipe_overview.md|DataHub Recipe Overview>
0 button
0 button
Hint: Mention @runllm in the thread for followups.
Hi <@U08PVDCPJAG>, how did my answer work out for you? Need any more assistance? You can also tag <@U07FV3UHE4D> to let your fellow community members know you’re still looking for guidance.