AKS - Canary does work as expected

[Leonardo-Ferreira]

What happened:

We setup a canary following the basics of https://kubernetes.github.io/ingress-nginx/examples/canary/ and only 3 things were different:

1. The original deployment has 5 replicas
2. I used the value 20 on the weight
3. The canary deployment has 2 replicas

Logs show that 40% of the traffic is directed to the canary deployment

It worked as expected on the testing environment, but there, both the original deployment and the canary, had one replica

What you expected to happen:

Using the value 20, we expected that 20% of the traffic to hit the canary deployment

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller
  Release:       v1.1.0
  Build:         cacbee86b6ccc45bde8ffc184521bed3022e7dee
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.9

Kubernetes version (use kubectl version): 1.26.6

Environment:

• Cloud provider or hardware configuration: AKS
• OS (e.g. from /etc/os-release): Ubuntu
• Kernel (e.g. uname -a):
• Install tools: N/A
• Basic cluster related info:
  • kubectl version: 1.28.8
  • kubectl get nodes -o wide

aks-nodepools01-27970517-vmss000000   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepools01-27970517-vmss000001   Ready    agent   20d   v1.26.6   172.xxx  <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepools01-27970517-vmss00000d   Ready    agent   27h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepools01-27970517-vmss00000e   Ready    agent   27h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss000001   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000a   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000c   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000h   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000m   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000p   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss00000r   Ready    agent   20d   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss000012   Ready    agent   27h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss000013   Ready    agent   27h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss000014   Ready    agent   26h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1
aks-nodepoolu01-27970517-vmss000015   Ready    agent   26h   v1.26.6   172.xxx   <none>        Ubuntu 22.04.3 LTS   5.15.0-1051-azure   containerd://1.7.5-1

• How was the ingress-nginx-controller installed:

helm ls -A | grep -I ingress
nginx-ingress                 	ingress-basic 	1       	2021-12-28 22:08:25.770338611 +0000 UTC	deployed	ingress-nginx-4.0.13                                                    	1.1.0      

• Current State of the controller:
  Name: nginx
  Labels: app.kubernetes.io/component=controller
  app.kubernetes.io/instance=nginx-ingress
  app.kubernetes.io/managed-by=Helm
  app.kubernetes.io/name=ingress-nginx
  app.kubernetes.io/version=1.1.0
  helm.sh/chart=ingress-nginx-4.0.13
  Annotations: meta.helm.sh/release-name: nginx-ingress
  meta.helm.sh/release-namespace: ingress-basic
  Controller: k8s.io/ingress-nginx
  Events:

• Current state of ingress object, if applicable:
  N/A

• Others:
  N/A

How to reproduce this issue:
Have more than 1 replica on the canary

Anything else we need to know:

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

/remove-kind bug

You can help a developer know more by posting outputs of commands like kubectl describe ..... of all the related resources and objects, in addition to the actual report showing 40%. I addition to the curl command as used.

You may also want to upgrade the controller to the latest release as the one reported is really old . If you do, then kindly edit the description accordingly.

There are multiple other questions asked in a new issue template. Providing that info helps in reducing guesses on your use case.