Troubleshooting DataHub Deployment in Amazon EKS with AWS EBS CSI Driver Installation

Original Slack Thread

Hi all! I’m attempting to deploy DataHub in Amazon Elastic Kubernetes Services, but I keep encountering the same error after installing the datahub/datahub-prerequisites. Has anyone encountered this before?

Hey <@U05JC2A6DMZ> , I am just encountering this now

AWS EBS CSI driver needs to provision EBS volumes for PVCs

kubectl get pods -n kube-system | grep ebs-csi

assuming you don’t see any output from that, you might not have the AWS EBS Container Storage Interface driver installed either

Ah, yeah. There wasn’t any output from that. Good catch.

This should be in the AWS EKS deployment guide imo

I agree. Thanks for the guidance, <@U05U4UUKULD>!

<@U05U4UUKULD> Do you know how long it took for the prerequisites to be installed? When running kubectl get pods after installing the AWS EBS Container Storage Interface plugin, I still see "pending."

Darn. Installing that plugin didn’t seem to work.

Hey Colin - I installed the plugin as well and it seems afterwards you have to ensure the EBS CSI has the correct policies to create EC2 volumes

It might be something else as well but the above was the first hiccup I ran into which was resolved:

https://medium.com/@donovanmaree/resolving-aws-eks-node-issues-authorizing-the-ebs-csi-in-datahub-deployment-01c849b35ae1

Brilliant. Thanks for the tip, <@U05U4UUKULD>!

<@U05JC2A6DMZ> running into the same issue now

Did the previous solution help you ?

<@U0626QUJEKX> I actually switched to a GCP deployment. It was easier to complete admittedly.

Oh okay. Thanks for the heads up. Yesterday I got to a point where I could get the front end locally . Today when trying to rerun the scripts my pods just got stuck in a pending status. Try to terminate it, and now it has been stuck in terminating mode. <@U05U4UUKULD> any insight

<@U0626QUJEKX> Send me your pod logs over dm if this is still an issue :slightly_smiling_face:

Hi <@U05U4UUKULD> thanks for reply. I sorted it out. I will share images of future errors next time .

<@U05U4UUKULD> <@U05JC2A6DMZ> Hi am using the quick start guide to create demo project on datahub. https://datahubproject.io/docs/deploy/aws. I am stuck here currently “Note down the elb address in the address column. Add the DNS CNAME record to the host domain pointing the host-name ( from above) to the elb address. DNS updates generally take a few minutes to an hour. Once that is done, you should be able to access datahub-frontend through the host-name.” Currently i don’t have a domain name . But i do have values

<@U0626QUJEKX>

Looks like you are busy in the Ingress section to expose the datahub-fronted.
In Kubernetes, an Ingress is an API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination, and name-based virtual hosting.

Ingress typically requires a domain name to function correctly because it relies on DNS to route traffic to the appropriate service within your cluster.
Without a domain name, you won’t be able to use standard DNS to resolve and route traffic to your Ingress controller and then to your services. ie access the Frontend.

If you are still in testing:

• Temporary DNS: If you don’t have a domain, you can use a free temporary DNS service that provides you with a subdomain to point to your ELB. This is not recommended for production but can work for a demo or testing purposes.
• Hosts File: For local testing, you can edit your local hosts file to point a domain (which you make up) to the ELB’s IP address. This change only affects your local machine and is not a scalable solution.
If you are this far in the process of testing out the load balancing and you are at the final steps of setting up Ingress, it might be worth investing in a cheap domain to continue exposing the Frontend.

I believe you can still run the ELB without ingress as well, but its more involved from where you are right now.

<@U05U4UUKULD> THANK YOU!!!

<@U05U4UUKULD> I appreciate the detail explaination . Now I really get it more . This was very helpful.