Troubleshooting Datahub Deployment on AWS EKS with Helm Install Prerequisites

Original Slack Thread

Hey all, I’m just getting started with Datahub trying to kick the tires but cant even get it installed. Im using the quick start for AWS EKS. I get to the step to helm install prerequisites and the pods stay pending forever. (

kubectl create secret generic neo4j-secrets --from-literal=neo4j-password=placeholder

helm repo add datahub <>

helm install prerequisites datahub/datahub-prerequisites```

kubectl get pods
elasticsearch-master-0 0/1 Pending 0 18h
prerequisites-kafka-0 0/1 Pending 0 18h
prerequisites-mysql-0 0/1 Pending 0 18h
prerequisites-zookeeper-0 0/1 Pending 0 18h```

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!
Name:             prerequisites-mysql-0
Namespace:        default
Priority:         0
Service Account:  prerequisites-mysql
Node:             <none>
Annotations:      checksum/configuration: f4319d4718b248202e23fec5a6be3eedfab5c74178052f0ef5ce1fd05b945270
Status:           Pending
IPs:              <none>
Controlled By:    StatefulSet/prerequisites-mysql
    Port:       3306/TCP
    Host Port:  0/TCP
    Liveness:   exec [/bin/bash -ec password_aux="${MYSQL_ROOT_PASSWORD:-}"
if [[ -f "${MYSQL_ROOT_PASSWORD_FILE:-}" ]]; then
    password_aux=$(cat "$MYSQL_ROOT_PASSWORD_FILE")
mysqladmin status -uroot -p"${password_aux}"
] delay=5s timeout=1s period=10s #success=1 #failure=3
    Readiness:  exec [/bin/bash -ec password_aux="${MYSQL_ROOT_PASSWORD:-}"
if [[ -f "${MYSQL_ROOT_PASSWORD_FILE:-}" ]]; then
    password_aux=$(cat "$MYSQL_ROOT_PASSWORD_FILE")
mysqladmin status -uroot -p"${password_aux}"
] delay=5s timeout=1s period=10s #success=1 #failure=3
    Startup:  exec [/bin/bash -ec password_aux="${MYSQL_ROOT_PASSWORD:-}"
if [[ -f "${MYSQL_ROOT_PASSWORD_FILE:-}" ]]; then
    password_aux=$(cat "$MYSQL_ROOT_PASSWORD_FILE")
mysqladmin status -uroot -p"${password_aux}"
] delay=15s timeout=1s period=10s #success=1 #failure=10
      BITNAMI_DEBUG:        false
      MYSQL_ROOT_PASSWORD:  <set to the key 'mysql-root-password' in secret 'mysql-secrets'>  Optional: false
      MYSQL_DATABASE:       my_database
      /bitnami/mysql from data (rw)
      /opt/bitnami/mysql/conf/my.cnf from config (rw,path="my.cnf")
      /var/run/secrets/ from kube-api-access-g79gw (ro)
  Type           Status
  PodScheduled   False 
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  data-prerequisites-mysql-0
    ReadOnly:   false
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      prerequisites-mysql
    Optional:  false
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:        op=Exists for 300s
                    op=Exists for 300s
  Type     Reason            Age                    From               Message
  ----     ------            ----                   ----               -------
  Warning  FailedScheduling  6m44s (x111 over 18h)  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition```

I believe theres something going on with the storageclasses/volumeattachment

I figured out the quickstart is missing a lot. It says nothing about enabling ebs,networking drivers and required ebs permissions

<@U06G9PR5Q80> Could you please share us the resolution? MTY

I installed the following add-ons:

  1. CoreDNS
  2. Kube-proxy
  3. EBS CSI Driver
  4. VPC CNI

Im trying to confirm this is all I did, but I’m failing to install it again.

I locked myself out attempting the front-end LB

I got it running by only installing EBS CSI. I had to give the nodegroup role more permissions. I gave it admin but i suspect it only needs EBS permissions.

NAME                                             READY   STATUS      RESTARTS   AGE
datahub-acryl-datahub-actions-759f74b7bf-gwd2q   1/1     Running     0          3m53s
datahub-datahub-frontend-66bcd79b95-dmbpc        1/1     Running     0          3m53s
datahub-datahub-gms-8fdfbc6c6-8456v              1/1     Running     0          3m53s
datahub-datahub-system-update-job-bzfpp          0/1     Completed   0          5m42s
datahub-elasticsearch-setup-job-95658            0/1     Completed   0          8m46s
datahub-kafka-setup-job-v5rgs                    0/1     Completed   0          8m40s
datahub-mysql-setup-job-wxtln                    0/1     Completed   0          5m48s
datahub-nocode-migration-job-jw7zr               0/1     Error       0          3m53s
datahub-nocode-migration-job-wlzmh               0/1     Completed   0          2m
elasticsearch-master-0                           1/1     Running     0          14m
prerequisites-kafka-0                            1/1     Running     0          14m
prerequisites-mysql-0                            1/1     Running     0          14m
prerequisites-zookeeper-0                        1/1     Running     0          14m```

got this warning when upgrading the install for ingress:
W0201 08:39:21.089400 32585 warnings.go:70] annotation “|” is deprecated, please use ‘spec.ingressClassName’ instead

I havent completed the database steps yet, but since adding the front end, the website is blank at login…

the documentation for front end says to use latest. I used an older version

  enabled: true
    repository: linkedin/datahub-frontend-react
    tag: "v0.10.4"```

so it works now

It works for me too, thank you :slightly_smiling_face: <@U06G9PR5Q80>