Creating Datahub Tokens and Handling Authentication Settings

troubleshoot
1

Original Slack Thread

Hi All!

Is the https://datahubproject.io/docs/api/graphql/token-management/|given graphql API the only API currently available to generate a datahub token via API call(programmatically)? I am deploying datahub on a k8s instance and by default, since metadata_service_authentication is true in https://github.com/acryldata/datahub-helm/blob/89c92c8ac73b4dc371d647216d60dff28cc7c9ae/charts/datahub/values.yaml#L699|helm-chart, the graphql API to create a PAT itself fails with a 401 as below.

Kindly let me know the answers to below queries

  1. Is there a way to create a token via API (not via Datahub UI) even if ``metadata_service_authentication` is set to true?
  2. If not, does that mean, I need to override metadata_service_authentication and set it to false, then create the token via the above graphql API, and if i want to use that token for other API requests to datahub, set metadata_service_authentication back to true and do helm upgrade and pass that token in further API calls ? Will that token be valid after helm upgrade?
2

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
3

![attachment]({‘ID’: ‘F0710R05H0U’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U06ASNNEHHC’, ‘CREATED’: ‘2024-04-27 03:43:22+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-04-27 03:43:22+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F0710R05H0U-3bb856b24b’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1714189406.565799’, ‘PARENT_MESSAGE_TS’: ‘1714189102.204499’, ‘MESSAGE_CHANNEL_ID’: ‘C029A3M079U’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 95155, ‘_FIVETRAN_SYNCED’: ‘2024-04-28 12:56:52.728000+00:00’})

4

I have not tested it myself, but once you have enabled metadata_service_authetication, don‘t you have to provide an Authorization header in each request; in this case with a token of an user which is allowed to manage tokens (e.g. the datahub root user)? So you basically you need a token to create a token…

Here the authorization header is mentioned in the example: https://datahubproject.io/docs/api/graphql/how-to-set-up-graphql#curl|https://datahubproject.io/docs/api/graphql/how-to-set-up-graphql#curl

In the guide for the access token management via GraphQL the authorization header is unfortunately missing in the examples…

5

Technically, instead of a token you could also use the system client id and secret (Basic auth. instead of Bearer): https://github.com/datahub-project/datahub/blob/3ab4ec9b449c5f35e46636f388b85783e5932db0/metadata-ingestion/src/datahub/cli/cli_utils.py#L73|https://github.com/datahub-project/datahub/blob/3ab4ec9b449c5f35e46636f388b85783e5932db0/metadata-ingestion/src/datahub/cli/cli_utils.py#L73

With the system client id I mean „__datahub_system“, the secret is stored in one of the K8s secrets…not sure whether it is recommended to use it in this case (you cannot easily revoke this secret like you can revoke tokens and I am not sure what happens if you change it in the K8s secret)…

https://github.com/acryldata/datahub-helm/blob/89c92c8ac73b4dc371d647216d60dff28cc7c9ae/charts/datahub/templates/datahub-auth-secrets.yml#L19|https://github.com/acryldata/datahub-helm/blob/89c92c8ac73b4dc371d647216d60dff28cc7c9ae/charts/datahub/templates/datahub-auth-secrets.yml#L19

6

<@U03GWPR6FSS> I tried to pass the clientId and client secret as Basic auth (instead of using Bearer token) for the token generation via GraphQL API, and still receives a 401. Could you kindly confirm whether using Basic auth with client id and client secret indeed is supported for API requests to Datahub?

7

The values for client id and client secret, just for now, i took from the envs in pods themselves , so i believe the values are indeed right what i passed![attachment]({‘ID’: ‘F070W0PEAH3’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U06ASNNEHHC’, ‘CREATED’: ‘2024-04-27 11:10:23+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-04-27 11:10:23+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F070W0PEAH3-f54e8e43dd’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1714216229.470089’, ‘PARENT_MESSAGE_TS’: ‘1714189102.204499’, ‘MESSAGE_CHANNEL_ID’: ‘C029A3M079U’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 8698, ‘_FIVETRAN_SYNCED’: ‘2024-04-28 12:56:52.881000+00:00’})![attachment]({‘ID’: ‘F0711BX1PGC’, ‘EDITABLE’: False, ‘IS_EXTERNAL’: False, ‘USER_ID’: ‘U06ASNNEHHC’, ‘CREATED’: ‘2024-04-27 11:09:05+00:00’, ‘PERMALINK’: ‘Slack’, ‘EXTERNAL_TYPE’: ‘’, ‘TIMESTAMPS’: ‘2024-04-27 11:09:05+00:00’, ‘MODE’: ‘hosted’, ‘DISPLAY_AS_BOT’: False, ‘PRETTY_TYPE’: ‘PNG’, ‘NAME’: ‘image.png’, ‘IS_PUBLIC’: True, ‘PREVIEW_HIGHLIGHT’: None, ‘MIMETYPE’: ‘image/png’, ‘PERMALINK_PUBLIC’: ‘https://slack-files.com/TUMKD5EGJ-F0711BX1PGC-eeaad9caa1’, ‘FILETYPE’: ‘png’, ‘EDIT_LINK’: None, ‘URL_PRIVATE’: ‘Slack’, ‘HAS_RICH_PREVIEW’: False, ‘TITLE’: ‘image.png’, ‘IS_STARRED’: False, ‘PREVIEW_IS_TRUNCATED’: None, ‘URL_PRIVATE_DOWNLOAD’: ‘Slack’, ‘PREVIEW’: None, ‘PUBLIC_URL_SHARED’: False, ‘MESSAGE_TS’: ‘1714216229.470089’, ‘PARENT_MESSAGE_TS’: ‘1714189102.204499’, ‘MESSAGE_CHANNEL_ID’: ‘C029A3M079U’, ‘_FIVETRAN_DELETED’: False, ‘LINES_MORE’: None, ‘LINES’: None, ‘SIZE’: 40240, ‘_FIVETRAN_SYNCED’: ‘2024-04-28 12:56:52.881000+00:00’})

8

<@U03BEML16LB> would you be able to help advise on this use-case? - If initial PAT (Bearer token) can be created only via UI, is there some kind of support for Basic auth? Can this basic auth be used for other rest APIs to ingest and search for data?

cc <@U06CMSR2S1E> <@U02RXEEJQBH>