Troubleshooting Datahub Python Client Error when Emitting Events

Original Slack Thread

hi everyone and datahub team ,

I encounter an error with python client acryl-datahub==0.11.0 . everything’s fine except emitting events to gms with the said python lib.

the below script in python


graph.test_connection()```
returns:
```{'models': {},
 'patchCapable': True,
 'versions': {'linkedin/datahub': {'version': 'v0.11.0',
   'commit': '68ae3bfc26837f9b26534e0bf565c1ad7fd0d252'}},
 'managedIngestion': {'defaultCliVersion': '0.11.0', 'enabled': True},
 'statefulIngestionCapable': True,
 'supportsImpactAnalysis': True,
 'timeZone': 'GMT',
 'telemetry': {'enabledCli': True, 'enabledIngestion': False},
 'datasetUrnNameCasing': False,
 'retention': 'true',
 'datahub': {'serverType': 'prod'},
 'noCode': 'true'}```
but when I
```graph.emit(event)```
it returns
``` ('Unable to emit metadata to DataHub GMS', {'message': '401 Client Error: Unauthorized for url: http://<main-url>/aspects?action=ingestProposal'})```
env. variable
`DATAHUB_DEBUG`  is set to true in the gms pod. But I wonder which piece of info. I should take a look at while troubleshoot.

helm chart version `0.2.188`  is used
aws opensearch version is at `1.3`  (if needed)



Appreciate everyone’s assistance. thanks

hi <@UV5UEC3LN>,
may i know whether you could help ? thanks in advance

while the gms pod is initialised , there is log below , I am not sure whether it’s relevant.

│ java.io.IOException: Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=https://&lt;opensearch-endpoint&gt;.<http://ap-southeast-1.es.amazonaws.com:443|ap-southeast-1.es.amazonaws.com:443>, response=HTTP/1.1 200 OK}                │
│     at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1783)                                                                                                                                                             │
│     at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:636)                                                                                                                                                  │
│     at org.elasticsearch.client.RestClient$1.completed(RestClient.java:376)                                                                                                                                                                                │
│     at org.elasticsearch.client.RestClient$1.completed(RestClient.java:370)                                                                                                                                                                                │
│     at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)                                                                                                                                                                              │
│     at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)                                                                                                                       │
│     at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)                                                                                                                                            │
│     at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)                                                                                                                                                 │
│     at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)                                                                                                                                           │
│     at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)                                                                                                                                                         │
│     at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)                                                                                                                                                         │
│     at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:121)                                                                                                                                                         │
│     at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)                                                                                                                                                                     │
│     at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)                                                                                                                                                         │
│     at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)                                                                                                                                                        │
│     at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)                                                                                                                                                              │
│     at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)                                                                                                                                                                      │
│     at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)                                                                                                                                     │
│     at java.base/java.lang.Thread.run(Thread.java:829)                                                                                                                                                                                                     │
│ Caused by: java.lang.NullPointerException: null                                                                                                                                                                                                            │
│     at java.base/java.util.Objects.requireNonNull(Objects.java:221)                                                                                                                                                                                        │
│     at org.elasticsearch.action.DocWriteResponse.&lt;init&gt;(DocWriteResponse.java:127)                                                                                                                                                                         │
│     at org.elasticsearch.action.update.UpdateResponse.&lt;init&gt;(UpdateResponse.java:65)                                                                                                                                                                       │
│     at org.elasticsearch.action.update.UpdateResponse$Builder.build(UpdateResponse.java:172)                                                                                                                                                               │
│     at org.elasticsearch.action.update.UpdateResponse$Builder.build(UpdateResponse.java:160)                                                                                                                                                               │
│     at org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:159)                                                                                                                                                              │
│     at org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:188)                                                                                                                                                                      │
│     at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911)                                                                                                                                                             │
│     at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$10(RestHighLevelClient.java:1699)                                                                                                                             │
│     at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1781)                                                                                                                                                             │
│     ... 18 common frames omitted```

Issue fixed after i re-generate pat with non root user on UI.

Surprised the pat generated by root or non-root user makes that huge difference and there no log catching this error or doc mentioned it. :melting_face:

The error is returned to you. You are getting a 401 unauthorized response because the user did not have proper permissions to access the ingestProposal endpoint. When you generated the personal access token with the admin user, the token has full permissions. Personal access tokens are personal, the token gets the access level of the user generating it.

thanks for your reply <@UV5UEC3LN>
as i said earlier ,
> Issue fixed after i re-generate pat with non root user on UI.
I assumed the token has full permission in default. but it seems that it’s not.
I resort to non-admin user to re-generate the PAT to overcome it.