Adding METADATA_SERVICE_AUTH_ENABLED Environment Variable in DataHub Docker Compose File

ingestion
1

Original Slack Thread

i went to the datahub documentation and got the docker compose file from there, but there is also no option for environment variables there

https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose-without-neo4j-m1.quickstart.yml

as you can see, in this compose file, there is no METADATA_AUTH_ENABLE variable

2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

<@U06TM7M40RK> Can you help with the questions in this thread?

:information_source: _Please continue to tag RunLLM (e.g <@U06TM7M40RK>) to have it continue generating responses!

3

To enable token-based authentication in DataHub using the provided Docker Compose file, you need to add the METADATA_SERVICE_AUTH_ENABLED environment variable manually. Here are the steps to modify your docker-compose-without-neo4j-m1.quickstart.yml file:

  1. Edit docker-compose-without-neo4j-m1.quickstart.yml File:
    • Open the docker-compose-without-neo4j-m1.quickstart.yml file.
    • Add the METADATA_SERVICE_AUTH_ENABLED environment variable to both the datahub-gms and datahub-frontend services.

Here is an example of how to do this:

version: '3.5'
services:
  datahub-gms:
    image: linkedin/datahub-gms
    environment:
      METADATA_SERVICE_AUTH_ENABLED: "true"
    ports:
      - "8080:8080"

  datahub-frontend:
    image: linkedin/datahub-frontend
    environment:
      METADATA_SERVICE_AUTH_ENABLED: "true"
    ports:
      - "9002:9002"
  1. Restart Docker Services:
    • After modifying the file, restart the services to apply the changes.
docker-compose down
docker-compose up -d

  1. Generate and Use a Personal Access Token:

    • After enabling authentication, generate a Personal Access Token (PAT) via the DataHub UI.
    • Navigate to Settings > Access Tokens > Generate Personal Access Token.
    • Save the generated token securely.

  2. Update Ingestion Recipes:

    • Update your ingestion recipes to include the generated token.
source:
  # source configs
sink:
  type: "datahub-rest"
  config:
    server: 'http://datahub-gms:8080'
    token: <your-personal-access-token-here!>
  1. Use the Token in API Requests:
    • Include the token in the Authorization header for API requests.
curl 'http://localhost:9002/api/gms/entities/urn:li:corpuser:datahub' -H 'Authorization: Bearer <access-token>'

Would you like more detailed steps or assistance with a specific part of this process?

Sources:

  • <Slack Discussion> :+1: 0 button :-1: 0 button :bulb: Hint: Mention @runllm in the thread for followups.