Gateway API: unable to use multiple Gateways with a shared loadBalancerIP on GKE
#54453
Comments
vpedosyuk
changed the title
Gateway with a shared loadBalancerIP on GKEGateways with a shared loadBalancerIP on GKE
This is not how Gateway API is expected to be used. Gateway represents the physical infrastructure (an IP address, a real pod serving the traffic, etc). The expectation is sharing the same GW across multiple namespaces its done via routes and Obviously thats not to say it couldn't potentially be done by some implementations or with tweaks to those... but its really not the intended usage of the API |
|
@howardjohn, thanks, but it's somewhat confusing... I guess both of my Actually, I was happy that it's now easy to isolate heavy flows from regular 80/443 HTTP services, so that Kafka will have its own Envoy with its own autoscaling mechanisms attached. If Ana overloads Kafka with some tests, Chihiro wouldn't need to explain why Allison's web app is affected. And the fact that Istio exposes the |
|
Does GCP not create a conflict if there is no overlap? and it will handle forwarding to the correct envoy instance by the port? We could add an option to not expose the readiness endpoint. It's primarily added to handle some (non GCP) load balancers who perf health checks based on the first port in the service. |
|
According to the GCP docs (and to my experience as well) there shouldn't be any conflict:
To double-check that I just reproduced the original behavior once again: $ kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: one-gateway
namespace: istio-ingress
spec:
gatewayClassName: istio
addresses:
- value: 34.165.57.128 # my second reserved GCP Static IP
type: IPAddress
listeners:
- name: default
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
EOF
gateway.gateway.networking.k8s.io/one-gateway created
$ kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: two-gateway
namespace: kafka
spec:
gatewayClassName: istio
addresses:
- value: 34.165.57.128 # my second reserved GCP Static IP
type: IPAddress
listeners:
- name: kafka
port: 9093
protocol: TCP
allowedRoutes:
namespaces:
from: All
EOF
gateway.gateway.networking.k8s.io/two-gateway createdBut this time I also added two custom $ kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: one-gateway-custom-svc
namespace: istio-ingress
spec:
loadBalancerIP: 34.165.87.186 # my third reserved GCP Static IP
type: LoadBalancer
ports:
- name: default
port: 80
protocol: TCP
targetPort: 80
selector:
gateway.networking.k8s.io/gateway-name: one-gateway
EOF
service/one-gateway-custom-svc created
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: two-gateway-custom-svc
namespace: kafka
spec:
loadBalancerIP: 34.165.87.186 # my third reserved GCP Static IP
type: LoadBalancer
ports:
- name: default
port: 9093
protocol: TCP
targetPort: 9093
selector:
gateway.networking.k8s.io/gateway-name: two-gateway
EOF
service/two-gateway-custom-svc createdAs a result, my custom $ kubectl get svc -n istio-ingress one-gateway-istio one-gateway-custom-svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
one-gateway-istio LoadBalancer 34.118.237.195 34.165.57.128 15021:30225/TCP,80:30250/TCP 4m7s
one-gateway-custom-svc LoadBalancer 34.118.232.95 34.165.87.186 80:30130/TCP 2m43s
$ kubectl get svc -n kafka two-gateway-istio two-gateway-custom-svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
two-gateway-istio LoadBalancer 34.118.236.129 <pending> 15021:32173/TCP,9093:30457/TCP 3m34s
two-gateway-custom-svc LoadBalancer 34.118.229.51 34.165.87.186 9093:30243/TCP 2m17s |
|
And I forgot to say that the option to not expose the readiness endpoint sounds awesome! |
|
@howardjohn I've created a PR with the new option that resolves the issue. Your feedback would be greatly appreciated. |
Is this the right place to submit this?
Bug Description
I have a GKE cluster where I need to expose multiple services, some are HTTP, some are TCP. I'm trying to achieve that with Istio and the new Gateway API architecture. However, I'd like all these services to have the same external cloud IP address - I believe that's a quite common use-case.
So I'm creating 2
Gatewayresources:one-gatewayfor HTTP traffic in theistio-ingressnamespace.two-gatewayfor TCP traffic in thekafkanamespace.As documented, Istio behaves a bit differently with Gateway API, i.e. it creates a separate "ingress" pod for each
Gatewayalong with a dedicatedServiceresource of typeLoadBalancerin respective namespaces.The problem is that each auto-created
Serviceexposes15021port by default and when configured to use a sharedloadBalancerIP, one of the services just fails to boot up and reports "conflicting ports". AFAIK, there's no way to workaround that.Reproducing the issue on a dummy GKE cluster:
And now checking the result:
Once I remove the
two-gatewaygateway and include the needed port into the configuration ofone-gateway, it works fine:P.S. I'm referring to GCP as that's what I'm using currently but I think the problem is relevant for other cloud providers as well.
Version
Additional Information
No response
The text was updated successfully, but these errors were encountered: