Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingress Time outs after 2 minutes even though time out is set to 3600s #10735

Open
kherath17 opened this issue Dec 7, 2023 · 4 comments
Open

Ingress Time outs after 2 minutes even though time out is set to 3600s #10735

kherath17 opened this issue Dec 7, 2023 · 4 comments
Assignees
@strongjz
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@kherath17
Copy link

What happened:
I have setup an ingress service to route requests to Selenium Grid on K8, even though the requests are reached to grid que, it timeouts after 2 minutes.

Returned below response and execution stopped
<head><title>504 Gateway Time-out</title></head>
<body>
<center>504 Gateway Time-out</center>
</body>
</html>

What you expected to happen:
Ingress should honor the timeout values we have configured through below annotations :
nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.): v1.7.0

Kubernetes version (use kubectl version): v1.28.1

Environment:

  • Cloud provider or hardware configuration: AWS

  • OS (e.g. from /etc/os-release): linux

  • Kernel (e.g. uname -a):

  • Install tools:

    • Please mention how/where was the cluster created like kubeadm/kops/minikube/kind etc.

  • Basic cluster related info:

    • kubectl version
    • kubectl get nodes -o wide

  • How was the ingress-nginx-controller installed:

    • If helm was used then please show output of helm ls -A | grep -i ingress
      ->nginx-ingress-release default 1 2023-04-22 17:30:21.803047 +0530 +0530 deployed ingress-nginx-4.4.2 1.5.1
    • If helm was used then please show output of helm -n <ingresscontrollernamepspace> get values <helmreleasename>
    • If helm was not used, then copy/paste the complete precise command used to install the controller, along with the flags and options used
    • if you have more than one instance of the ingress-nginx-controller installed in the same cluster, please provide details for all the instances

  • Current State of the controller:

    • kubectl describe ingressclasses
      Name: nginx
      Labels: app.kubernetes.io/component=controller
      app.kubernetes.io/instance=nginx-ingress-release
      app.kubernetes.io/managed-by=Helm
      app.kubernetes.io/name=ingress-nginx
      app.kubernetes.io/part-of=ingress-nginx
      app.kubernetes.io/version=1.5.1
      helm.sh/chart=ingress-nginx-4.4.2
      Annotations: ingressclass.kubernetes.io/is-default-class: true
      meta.helm.sh/release-name: nginx-ingress-release
      meta.helm.sh/release-namespace: default
      Controller: k8s.io/ingress-nginx
      Events: none
    • kubectl -n <ingresscontrollernamespace> get all -A -o wide
    • kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>
    • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>

  • Current state of ingress object, if applicable:

    • kubectl -n <appnnamespace> get all,ing -o wide
    • kubectl -n <appnamespace> describe ing <ingressname>
    • If applicable, then, your complete and exact curl/grpcurl command (redacted if required) and the reponse to the curl/grpcurl command with the -v flag

  • Others:

    • Any other related information like ;
      • copy/paste of the snippet (if applicable)
      • kubectl describe ... of any custom configmap(s) created and in use
      • Any other related information that may help

How to reproduce this issue:
Setup Selenium Grid on K8 including below ingress and when requests are routed to Selenium Grid it should honor the timeouts mentioned

Sample Ingress File

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: qlabv2-ingress
annotations:
`kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
rules:

  • http:
    paths:
  • path: /sandbox_qlabv2(/|$)(.)
    pathType: Prefix
    backend:
    service:
    name: selenium-hub-service
    port:
    number: 80
  • path: /(/?)(session/./se/vnc)
    pathType: Prefix
    backend:
    service:
    name: selenium-hub-service
    port:
    number: 80

Anything else we need to know:
@kherath17 kherath17 added the kind/bug Categorizes issue or PR as related to a bug. label Dec 7, 2023
@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Dec 7, 2023
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kherath17 kherath17 mentioned this issue Dec 7, 2023
Closed
@strongjz
Copy link
Member

strongjz commented Dec 7, 2023

Which load balancer did you deploy in EKS?
Do you have endpoints for the service?

@strongjz
Copy link
Member

strongjz commented Dec 7, 2023

/assign @strongjz

@kherath17
Copy link
Author

@strongjz apologies for the delayed response

Which load balancer did you deploy in EKS? - Nginx Load Balancer
Do you have endpoints for the service? - We have shared end points with context paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@strongjz strongjz

Labels
kind/bug Categorizes issue or PR as related to a bug. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
[SIG Network] Ingress NGINX
Status: No status
Milestone
No milestone
Development

No branches or pull requests
3 participants
@strongjz @k8s-ci-robot @kherath17