Managing CLI Version Compatibility in Datahub Upgrade to 10.5

troubleshoot
1

Original Slack Thread

Hi, Yesterday we upgraded our Datahub to 10.5 and Presto ingest failed with this error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/datahub/entrypoints.py", line 175, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))```
Please help. Full log in a file attached

---
Update: I've tried running ingest with previos CLI version (0.10.4) and it worked.
So looks like the problem is with CLI 10.5?![attachment](https://files.slack.com/files-pri/TUMKD5EGJ-F061FG13VC6/error.log?t=xoxe-973659184562-6705490291811-6708051934148-dd1595bd5f63266bc09e6166373c7a3c)
2

The cli version is quite old -> cli_version": "0.10.0.6
based on the logs.

3

<@UV14447EU> hmm, isn’t CLI version supposed to be updated along with release update automatically?

4

If you run your ingestion from the UI then it should except if you pinned your client in your ingestion ->
https://datahubproject.io/docs/ui-ingestion/#advanced-running-with-a-specific-cli-version

5

<@UV14447EU> I run ingestions from UI, yes. The CLI versions weren’t pinned when errors occured. So our Datahub was just upgraded to 10.5, but it had CLI 0.10.0.6 for ingests for some reason.
Is it a bug on Datahubs side, or shoud we have set it somewhere while upgrading?
We are deployed via Kubernetes and helm charts

6

which version of datahub-action do you have in your helm chart ->
https://github.com/acryldata/datahub-helm/blob/4a19805ba95af9fbb349fc837d37b88be3daff6d/charts/datahub/values.yaml#L55

7

<@U03LYB2ESJ0> sorry, my previous answer was not correct as this is the config property in helm, which controls the default cli version of managed ingestion ->
https://github.com/acryldata/datahub-helm/blob/4a19805ba95af9fbb349fc837d37b88be3daff6d/charts/datahub/values.yaml#L601

8

<@UV14447EU> defaultCliVersion: "0.10.5.4"

9

then if it is not pinned it should use the defaultCliVersion

10

Well it didn’t happen like that:woman-shrugging:
Something in my head rings a bell that it’s not the first time such thing happened among Datahub users. Maybe it’s worth checking on your side, as smth that looks like a bug?

11

well, I’m not super familiar with the helm side but I can ask around internally if somebody has seen an issue like this

12

Nice, thank you!

13

A few things to check:
• In GMS (or MAE Consumer in standalone mode) the environment variable UI_INGESTION_DEFAULT_CLI_VERSION should show the correct version in the deployment, can you double check this?
• When does your actions pod show it was last restarted? Did it restart with the rest of your deployments with the update?
• Query your ingestion sources and see what the input argument for the CLI Version was using this query:

  listIngestionSources(input: $input) {
    ingestionSources {
      urn
      name
      type
      executions(start: 0, count: 1) {
        executionRequests {
          urn
          id
          input {
            requestedAt
            arguments {
              value
            }
            __typename
          }
          result {
            status
            startTimeMs
            durationMs
            __typename
          }
          __typename
        }
        __typename
      }
      __typename
    }
    __typename
  }
}```

```variables:
{
"input": {}
}```
This should give the CLI version in the `value` field of arguments, the other fields are just included for identification purposes