Setting up a Fixed IP for Datahub Frontend in Azure using a Load Balancer

all-things-deployment
1

Original Slack Thread

Hi,

We have a working deployment of Datahub in Azure. We are now working with setting up a fixed IP for the frontend. To accomplish this, we are following the instructions in the documentation:

<Deploying to Azure | DataHub endpoints using a load balancer>

We are following these steps:

  1. Launch an AKS cluster
  2. Setup Datahub using helm (As in https://datahubproject.io/docs/deploy/kubernetes/ ). The relevant part of the values.yaml for datahub looks like this:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
  resources:
    limits:
      memory: 4200Mi
    requests:
      cpu: 400m
      memory: 4200Mi
  # Set up ingress to expose react front-end
  ingress:
    enabled: false```
Please note that this will be changed later on to enable the ingress.


3. Create WAF policy with:

```az network application-gateway waf-policy create -g myResourceGroup -n myWAFPolicy```
4. Create publicIP, Vnet, Subnet with:

```az network public-ip create -n myPublicIp -g myResourceGroup --allocation-method Static --sku Standard

az network vnet create -n myVnet -g myResourceGroup --address-prefix 10.0.0.0/16 --subnet-name mySubnet --subnet-prefix 10.0.0.0/24 

az network application-gateway create -n myApplicationGateway -l eastus -g myResourceGroup --sku WAF_v2 --public-ip-address myPublicIp --vnet-name myVnet --subnet mySubnet --priority 100 --waf-policy /subscriptions/{subscription_id}/resourceGroups/myResourceGroup/providers/Microsoft.Network/ApplicationGatewayWebApplicationFirewallPolicies/myWAFPolicy```
5. Enable AGIC Add-on on the newly created AKS cluster with:

```appgwId=$(az network application-gateway show -n myApplicationGateway -g myResourceGroup -o tsv --query "id") 
az aks enable-addons -n myCluster -g myResourceGroup -a ingress-appgw --appgw-id $appgwId```
6. Setting up peering between the two virtual networks

```nodeResourceGroup=$(az aks show -n myCluster -g myResourceGroup -o tsv --query "nodeResourceGroup")
aksVnetName=$(az network vnet list -g $nodeResourceGroup -o tsv --query "[0].name")

aksVnetId=$(az network vnet show -n $aksVnetName -g $nodeResourceGroup -o tsv --query "id")
az network vnet peering create -n AppGWtoAKSVnetPeering -g myResourceGroup --vnet-name myVnet --remote-vnet $aksVnetId --allow-vnet-access

appGWVnetId=$(az network vnet show -n myVnet -g myResourceGroup -o tsv --query "id")
az network vnet peering create -n AKStoAppGWVnetPeering -g $nodeResourceGroup --vnet-name $aksVnetName --remote-vnet $appGWVnetId --allow-vnet-access```
7. Edit the values.yaml for datahub so that it looks like this:

```datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
  resources:
    limits:
      memory: 4200Mi
    requests:
      cpu: 400m
      memory: 4200Mi
  # Set up ingress to expose react front-end
  ingress:
    enabled: true
    annotations:
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: azure/application-gateway
      <http://appgw.ingress.kubernetes.io/backend-protocol|appgw.ingress.kubernetes.io/backend-protocol>: "http" 
    hosts:
    - paths:
      - /*```
8. Apply the updates:

```helm upgrade --install datahub datahub/datahub --values values.yaml```

Now, the problem is that after everything runs successfully (or without getting any pod that fails, or any evident errors), when we run

```kubectl get ingress```
We get an ingress without an address:

```NAME                    CLASS    HOSTS   ADDRESS   PORTS   AGE
datahub-datahub-frontend   &lt;none&gt;   *              80      14m34s```
Using the IP of the front-end service like this:

`http://&lt;ip-address-frontend-service&gt;:9002`

Datahub's frontend is available (this IP is not fixed and changes with each deployment, therefore we are trying to setup the fixed ip by setting up the ingress).

However, when trying to go to:

`http://&lt;fixed-ip-created-and-used-by-gateway&gt;`


We get a "502 Bad gateway" error.

What could we be missing? Or how can I debug my deployment? Is the documentation for deploying to Azure outdated or wrong?

At the end of the day we want a Fixed IP. Is there any other solution?

The versions are:

Prerequisites-chart: 0.1.6
Datahub-chart: 0.3.19
Datahub version:  0.12.1
2

Hi Arturo! Apologies for the delay – if you’re still hitting this issue, I highly suggest joining us at the Community Marathon tomorrow! We have a variety of troubleshooting & support sessions scheduled throughout the day where we can give live help :slight_smile: https://datahubspace.slack.com/archives/CUMV92XRQ/p1705433341623299

3

I didn’t see this message until now :confused: