diff --git a/docs/guides/multi-cluster.md b/docs/guides/multi-cluster.md new file mode 100644 index 00000000..141e8fbd --- /dev/null +++ b/docs/guides/multi-cluster.md @@ -0,0 +1,51 @@ +# Recommended Multi-Cluster Architecture + +Here is a recommended multi-cluster architecture if you'd like to setup kubernetes service-to-service communications across multiple clusters. + +Suppose your organization would like to have one large infrastructure that have many clusters in one AWS account. It's recommended your infrastructure includes the following components: +- One manually created VPC Lattice service network, (it could be created by either AWS Console, CLI, CloudFormation, Terraform or any other tools) +- Create vpc service network associations between VPC Lattice service network and each config cluster's VPC and workload clusters' VPCs +- Multiple workload cluster(s), that are used to run application workload(s). Workload cluster(s) should only have following workloads related kubernetes objects: + - Application workload(s) (`Pod(s)`, `Deployment(s)` etc.) + - `Service(s)` for application workload(s) + - `ServiceExport(s)`, that export kubernetes application Service(s) to the "config cluster" +- One extra dedicated "config cluster", which acts as a "service network control plane" and it should include following kubernetes objects: + - One `Gateway` that has __same name__ as the manually created VPC Lattice service network name + - `ServiceImport(s)`, that reference to kubernetes application services that are exported from workload cluster(s) + - `HTTPRoute(s)`,`GRPCRoute(s)`, that have rules backendRefs to `ServiceImport(s)` that referring kubernetes application service(s) in workload cluster(s) + +![config cluster and multiple workload clusters](../images/multi-cluster.png) + + +Above architecture is recommended for following reasons: +- Putting all Route(s) and only one Gateway in the config cluster is easier to manage and have less chance to have conflict. +- Reduce blast radius. If there is any issue in the config cluster, it won't affect the applications in the workload cluster(s). +- It is easier to separate permissions and duties by different roles in your organization. For example: + - The infra operator role only has permissions and manages the resource in the config cluster and VPC Lattice resource, + - The application developer role only has permissions and manages the resource in the workload cluster(s). + + + +Following steps show you how to set up this recommended multi-cluster architecture with 1 config cluster and 2 workload clusters. +1. Create 3 k8s clusters: `config cluster`, `workload cluster-1`, `workload cluster-2`. Install AWS Gateway API Controller in each cluster, you could follow this instruction [deploy.md](deploy.md) +1. Create a VPC Lattice `ServiceNetwork` with name `my-gateway` +1. Create `VPCServiceNetworkAssociation(s)` between previous step created service network and each config cluster's VPC and workload clusters' VPCs +1. Setup following resource in the workload cluster1: + ``` + kubectl apply -f examples/service-1.yaml + kubectl apply -f examples/service-1-export.yaml + ``` +1. Setup following resource in the workload cluster2: + ``` + kubectl apply -f examples/service-2.yaml + kubectl apply -f examples/service-2-export.yaml + ``` +1. Setup following resource in the config cluster: + ``` + kubectl apply -f examples/my-gateway.yaml + kubectl apply -f examples/my-httproute.yaml + kubectl apply -f examples/service-1-import.yaml + kubectl apply -f examples/service-2-import.yaml + ``` +1. At this point, the connectivity setup finished, pods in workload cluster1 are able to communicate with `service-2` in workload cluster2 (and vice versa) via the `my-httproute` DNS name. +1. Furthermore, you could have more workload clusters to join the `my-gateway` service network by creating the `ServiceNewtorkAssociation(s)`, workloads there all be able to communicate with `service-1` and `service-2` via the `my-httproute` DNS name and path matching. diff --git a/docs/guides/multi-sn.md b/docs/guides/multi-sn.md deleted file mode 100644 index f4b8ef3f..00000000 --- a/docs/guides/multi-sn.md +++ /dev/null @@ -1,180 +0,0 @@ -# Multiple Gateways/Service Networks and HTTPRoute Associates to Multiple Gateways - -## Multiple Gateways/Service Networks Usecase - -Here is one popular multi-cluster architecture: - -- config cluster, where is used for configuration management -- multiple work-load cluster(s), where are used to run application workload(s) - -You can see a production usecase at AirBnb [airbnb mullti-cluster](https://www.youtube.com/watch?v=1D8lg36ZNHs) - -Here is our example - -![Config Cluster and multiple workload cluster](../images/multi-sn.png) - -- there are 2 gateway(s), gateway-1/lattice-service-network-1 and gateway-2/lattice-service-network-2 -- gateway-1 contains HTTPRoute1 and HTTPRoute2 -- gateway-2 contains HTTPRoute2 and HTTPRoute3 -- blue workload cluster(s) are using gateway-1 to access HTTPRoute1 an HTTPRoute2 -- orange workload cluster(s) are using gateway-2 to access HTTPRoute2 an HTTPRoute3 - -### Config Cluster Gateway Configuration - -``` -# gateway-1 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: gateway-1 - annotations: - application-networking.k8s.aws/lattice-vpc-association: "false" # not associate to VPC -spec: - gatewayClassName: amazon-vpc-lattice - listeners: - ... -``` - -``` -# gateway-2 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: gateway-2 - annotations: - application-networking.k8s.aws/lattice-vpc-association: "false" # not associate to VPC -spec: - gatewayClassName: amazon-vpc-lattice - listeners: - ... -``` - -``` -# httproute-1 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: httproute-1 -spec: - parentRefs: - - name: gateway-1 # part of gateway-1/service-network-1 - ... -``` - -``` -# httproute-2 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: httproute-2 -spec: - parentRefs: - - name: gateway-1 # part of both gateway-1 and gateway-2 - sectionName: http - - name: gateway-2 - sectionName: http - ... -``` - -``` -# httproute-3 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: -name: httproute-3 -spec: -parentRefs: -- name: gateway-2 # part of gateway-2/service-network-2 -... -``` - -### blue workload cluster(s) - -Associate cluster's VPC to gateway-1/service-network-1 so that all Pod(s) in blue workload clusters can access HTTPRoute(s)of gateway-1, HTTPRoute-1 and HTTPRoute-2 - -``` -# gateway-1 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: gateway-1 -spec: - gatewayClassName: amazon-vpc-lattice - listeners: - ... -``` - -### orange workload cluster(s) - -Associate cluster's VPC to gateway-2/service-network-2, so that all Pod(s) in orange workload clusters can access HTTPRoute(s) of gateway-2, HTTPRoute-2 an HTTPRoute-3 - -``` -# gateway-2 -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: gateway-2 -spec: - gatewayClassName: amazon-vpc-lattice - listeners: - ... -``` - -## Defining HTTPRoute in Config Cluster - -![ServiceImport](../images/serviceimport.png) - -### Exporting Kubernetes Service to AWS Lattice Service - -In workload cluster, exports k8s service to AWS VPC lattice - -``` -# in workload cluster(s) -apiVersion: application-networking.k8s.aws/v1alpha1 -kind: ServiceExport -metadata: - name: service-1 - annotations: - application-networking.k8s.aws/federation: "amazon-vpc-lattice" # AWS VPC Lattice -``` - -### Configure HTTPRoute in config cluster to reference K8S service(s) in worload cluster(s) - -``` -# in config cluster -apiVersion: application-networking.k8s.aws/v1alpha1 -kind: ServiceImport -metadata: - name: service-1 -spec: - type: ClusterSetIP - ports: - - port: 80 - protocol: TCP -``` - -``` -# httproute -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: httproute-1 -spec: - parentRefs: - - name: gateway-1 - sectionName: http - rules: - - backendRefs: - - name: service-1 - kind: ServiceImport - weight: 25 - - name: service-2 - kind: ServiceImport - weight: 25 - - name: service-3 - kind: ServiceImport - weight: 25 - - name: service-4 - kind: ServiceImport - weight: 25 -``` diff --git a/docs/images/GatewayUserGuideFigures.pptx b/docs/images/GatewayUserGuideFigures.pptx index 873f84eb..05c716aa 100644 Binary files a/docs/images/GatewayUserGuideFigures.pptx and b/docs/images/GatewayUserGuideFigures.pptx differ diff --git a/docs/images/multi-cluster.png b/docs/images/multi-cluster.png new file mode 100644 index 00000000..ec497e29 Binary files /dev/null and b/docs/images/multi-cluster.png differ diff --git a/docs/images/multi-sn.png b/docs/images/multi-sn.png deleted file mode 100644 index d40fd8c7..00000000 Binary files a/docs/images/multi-sn.png and /dev/null differ diff --git a/examples/my-gateway.yaml b/examples/my-gateway.yaml new file mode 100644 index 00000000..ec62fa7c --- /dev/null +++ b/examples/my-gateway.yaml @@ -0,0 +1,10 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: my-gateway +spec: + gatewayClassName: amazon-vpc-lattice + listeners: + - name: http + protocol: HTTP + port: 80 diff --git a/examples/my-httproute.yaml b/examples/my-httproute.yaml new file mode 100644 index 00000000..eef87395 --- /dev/null +++ b/examples/my-httproute.yaml @@ -0,0 +1,23 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: my-httproute +spec: + parentRefs: + - name: my-hotel + sectionName: http + rules: + - backendRefs: + - name: service-1 + kind: ServiceImport + matches: + - path: + type: PathPrefix + value: /service-1 + - backendRefs: + - name: service-2 + kind: ServiceImport + matches: + - path: + type: PathPrefix + value: /service-2 diff --git a/examples/service-1-export.yaml b/examples/service-1-export.yaml new file mode 100644 index 00000000..8732c379 --- /dev/null +++ b/examples/service-1-export.yaml @@ -0,0 +1,6 @@ +apiVersion: application-networking.k8s.aws/v1alpha1 +kind: ServiceExport +metadata: + name: service-1 + annotations: + application-networking.k8s.aws/federation: "amazon-vpc-lattice" diff --git a/examples/service-1-import.yaml b/examples/service-1-import.yaml new file mode 100644 index 00000000..909113f1 --- /dev/null +++ b/examples/service-1-import.yaml @@ -0,0 +1,9 @@ +apiVersion: application-networking.k8s.aws/v1alpha1 +kind: ServiceImport +metadata: + name: service-1 +spec: + type: ClusterSetIP + ports: + - port: 80 + protocol: TCP diff --git a/examples/service-1.yaml b/examples/service-1.yaml new file mode 100644 index 00000000..287cbb83 --- /dev/null +++ b/examples/service-1.yaml @@ -0,0 +1,36 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: service-1 + labels: + app: service-1 +spec: + replicas: 2 + selector: + matchLabels: + app: service-1 + template: + metadata: + labels: + app: service-1 + spec: + containers: + - name: service-1 + image: public.ecr.aws/x2j8p8w7/http-server:latest + env: + - name: PodName + value: "service-1 handler pod" + + +--- +apiVersion: v1 +kind: Service +metadata: + name: service-1 +spec: + selector: + app: service-1 + ports: + - protocol: TCP + port: 80 + targetPort: 8090 diff --git a/examples/service-2-export.yaml b/examples/service-2-export.yaml new file mode 100644 index 00000000..a1824db6 --- /dev/null +++ b/examples/service-2-export.yaml @@ -0,0 +1,6 @@ +apiVersion: application-networking.k8s.aws/v1alpha1 +kind: ServiceExport +metadata: + name: service-2 + annotations: + application-networking.k8s.aws/federation: "amazon-vpc-lattice" diff --git a/examples/service-2-import.yaml b/examples/service-2-import.yaml new file mode 100644 index 00000000..c975a463 --- /dev/null +++ b/examples/service-2-import.yaml @@ -0,0 +1,9 @@ +apiVersion: application-networking.k8s.aws/v1alpha1 +kind: ServiceImport +metadata: + name: service-2 +spec: + type: ClusterSetIP + ports: + - port: 80 + protocol: TCP diff --git a/examples/service-2.yaml b/examples/service-2.yaml new file mode 100644 index 00000000..ccc220da --- /dev/null +++ b/examples/service-2.yaml @@ -0,0 +1,36 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: service-2 + labels: + app: service-2 +spec: + replicas: 2 + selector: + matchLabels: + app: service-2 + template: + metadata: + labels: + app: service-2 + spec: + containers: + - name: service-2 + image: public.ecr.aws/x2j8p8w7/http-server:latest + env: + - name: PodName + value: "service-2 handler pod" + + +--- +apiVersion: v1 +kind: Service +metadata: + name: service-2 +spec: + selector: + app: service-2 + ports: + - protocol: TCP + port: 80 + targetPort: 8090