Fix release version, userdata, use of AMI ID in Launch Template

README.md

@@ -349,6 +349,7 @@ https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html
 | [aws_iam_role_policy_attachment.ipv6_eks_cni_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_launch_template.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) | resource |
 | [random_pet.cbd](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
+| [aws_ami.windows_ami](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
 | [aws_eks_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
 | [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.ipv6_eks_cni_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
@@ -363,8 +364,7 @@ https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html
 | <a name="input_additional_tag_map"></a> [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.<br>This is for some rare cases where resources want additional configuration of tags<br>and therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no |
 | <a name="input_after_cluster_joining_userdata"></a> [after\_cluster\_joining\_userdata](#input\_after\_cluster\_joining\_userdata) | Additional `bash` commands to execute on each worker node after joining the EKS cluster (after executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production | `list(string)` | `[]` | no |
 | <a name="input_ami_image_id"></a> [ami\_image\_id](#input\_ami\_image\_id) | AMI to use, overriding other AMI specifications, but must match `ami_type`. Ignored if `launch_template_id` is supplied. | `list(string)` | `[]` | no |
-| <a name="input_ami_release_version"></a> [ami\_release\_version](#input\_ami\_release\_version) | OBSOLETE: Use `ami_specifier` instead. Note that it has a different format.<br>Historical description: EKS AMI version to use, e.g. For AL2 \"1.16.13-20200821\" or for bottlerocket \"1.2.0-ccf1b754\" (no \"v\") or  for Windows \"2023.02.14\". For AL2, bottlerocket and Windows, it defaults to latest version for Kubernetes version." | `list(string)` | `[]` | no |
-| <a name="input_ami_specifier"></a> [ami\_specifier](#input\_ami\_specifier) | OS-dependent specifier for one of the several AMIs that match OS, architecture, and Kubernetes 1.xx version.<br>If not specified the recommended/latest AMI for the given Kubernetes version will be used.<br>Unfortunately, the format of this value varies by OS, and we have not found documentation for it.<br>You can generally figure it out from the AMI name or description, and validate it by trying to retrieve<br>the SSM Public Parameter for the AMI ID.<br><br>Examples:<br>  AL2: amazon-eks-node-1.29-v20240117<br>  AL2023: amazon-eks-node-al2023-x86\_64-standard-1.29-v20240605<br>  Bottlerocket: 1.20.1-7c3e9198  \_# Note: 1.20.1 is the Bottlerocket, not Kubernetes, version\_<br>  Windows: <not allowed> | `string` | `"recommended"` | no |
+| <a name="input_ami_release_version"></a> [ami\_release\_version](#input\_ami\_release\_version) | The EKS AMI "release version" to use. Defaults to the latest recommended version.<br>For Amazon Linux, it is the "Release version" from [Amazon AMI Releases](https://github.com/awslabs/amazon-eks-ami/releases)<br>For Bottlerocket, it is the release tag from [Bottlerocket Releases](https://github.com/bottlerocket-os/bottlerocket/releases) without the "v" prefix.<br>For Windows, it is "AMI version" from [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/eks-ami-versions-windows.html).<br>Note that unlike AMI names, release versions never include the "v" prefix.<br>Examples:<br>  AL2: 1.29.3-20240531<br>  Bottlerocket: 1.2.0 or 1.2.0-ccf1b754<br>  Windows: 1.29-2024.04.09 | `list(string)` | `[]` | no |
 | <a name="input_ami_type"></a> [ami\_type](#input\_ami\_type) | Type of Amazon Machine Image (AMI) associated with the EKS Node Group.<br>Defaults to `AL2_x86_64`. Valid values: `AL2_x86_64, AL2_x86_64_GPU, AL2_ARM_64, CUSTOM, BOTTLEROCKET_ARM_64, BOTTLEROCKET_x86_64, BOTTLEROCKET_ARM_64_NVIDIA, BOTTLEROCKET_x86_64_NVIDIA, WINDOWS_CORE_2019_x86_64, WINDOWS_FULL_2019_x86_64, WINDOWS_CORE_2022_x86_64, WINDOWS_FULL_2022_x86_64, AL2023_x86_64_STANDARD, AL2023_ARM_64_STANDARD`. | `string` | `"AL2_x86_64"` | no |
 | <a name="input_associate_cluster_security_group"></a> [associate\_cluster\_security\_group](#input\_associate\_cluster\_security\_group) | When true, associate the default cluster security group to the nodes. If disabled the EKS managed security group will not<br>be associated to the nodes and you will need to provide another security group that allows the nodes to communicate with<br>the EKS control plane. Be aware that if no `associated_security_group_ids` or `ssh_access_security_group_ids` are provided,<br>then the nodes will have no inbound or outbound rules. | `bool` | `true` | no |
 | <a name="input_associated_security_group_ids"></a> [associated\_security\_group\_ids](#input\_associated\_security\_group\_ids) | A list of IDs of Security Groups to associate the node group with, in addition to the EKS' created security group.<br>These security groups will not be modified. | `list(string)` | `[]` | no |
@@ -432,7 +432,6 @@ https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html
 
 | Name | Description |
 |------|-------------|
-| <a name="output_WARNING_ami_release_version"></a> [WARNING\_ami\_release\_version](#output\_WARNING\_ami\_release\_version) | Include the warning output message to quite the linter about unused variables. |
 | <a name="output_WARNING_cluster_autoscaler_enabled"></a> [WARNING\_cluster\_autoscaler\_enabled](#output\_WARNING\_cluster\_autoscaler\_enabled) | n/a |
 | <a name="output_eks_node_group_ami_id"></a> [eks\_node\_group\_ami\_id](#output\_eks\_node\_group\_ami\_id) | The ID of the AMI used for the worker nodes, if specified |
 | <a name="output_eks_node_group_arn"></a> [eks\_node\_group\_arn](#output\_eks\_node\_group\_arn) | Amazon Resource Name (ARN) of the EKS Node Group |

ami.tf

@@ -18,8 +18,12 @@
 
 
 locals {
+  given_ami_id = length(var.ami_image_id) > 0
+
   # Public SSM parameters all start with /aws/service/
 
+  ami_os = split("_", var.ami_type)[0]
+
   # format string that makes
   #    format(fmt, specifier, k8s_version) the SSM parameter name to retrieve
 
@@ -39,34 +43,74 @@ locals {
     WINDOWS_FULL_2022_x86_64   = "/aws/service/ami-windows-latest/Windows_Server-2022-English-Full-EKS_Optimized-%[2]v/image_id"
   }
 
-  # AMI specifiers?
-  # Specifiers for AL2 and AL2023 are AMI Name from https://github.com/awslabs/amazon-eks-ami/releases
+  release_version_parts              = concat(split("-", try(var.ami_release_version[0], "")), ["", ""])
+  amazon_linux_ami_name_release_part = try(join(".", slice(split(".", local.release_version_parts[0]), 0, 2)), "")
+  # AMI Public SSM Parameter specifiers?
+  # Release versions for AL2 and AL2023 are from https://github.com/awslabs/amazon-eks-ami/releases
+  # Amazon Linux Release Version: 1.29.0-20240213
   # AL2
   #   AMI name:      amazon-eks-node-1.29-v20240117
   #   AMI SSM param: /aws/service/eks/optimized-ami/1.29/amazon-linux-2/amazon-eks-node-1.29-v20240117/image_id
   # AL2023
-  #   AMI name:      amazon-eks-node-al2023-arm64-standard-1.29-v20240605
-  #   AMI SSM param: /aws/service/eks/optimized-ami/1.29/amazon-linux-2023/x86_64/standard/amazon-eks-node-al2023-x86_64-standard-1.29-v20240605/image_id
-  # Specifiers for Bottlerocket are the bare release version (e.g. `1.20.0`) or
-  #   the release version and the commit hash (e.g. `1.20.0-7c3e9198`)
+  #   AMI name:      amazon-eks-node-al2023-x86_64-standard-1.29-v20240213
+  #   AMI SSM param: /aws/service/eks/optimized-ami/1.29/amazon-linux-2023/x86_64/standard/amazon-eks-node-al2023-x86_64-standard-1.29-v20240213/image_id
+  # Specifiers for Bottlerocket are the bare release version (e.g. `1.18.0`) or
+  #   the release version and the first 8 characters of the commit hash (e.g. `1.18.0-7452c37e`). NOTE: GitHub commit hash abbreviations are only 7 characters.
+  # From:
   # Bottlerocket:
-  #   AMI name:      bottlerocket-aws-k8s-1.26-nvidia-x86_64-v1.17.0-53f322c2
-  #   AMI SSM param: /aws/service/bottlerocket/aws-k8s-1.26-nvidia/x86_64/1.17.0/image_id # No "v"
-  #      /aws/service/bottlerocket/aws-k8s-1.26-nvidia/x86_64/1.17.0--53f322c2/image_id
-  # Windows does not allow a specifier.
-  ami_specifier = var.ami_specifier == "recommended" && startswith(var.ami_type, "BOTTLEROCKET") ? "latest" : var.ami_specifier
-
-  # Kubernetes version priority (first one to be set wins)
-  # 1. var.kubernetes_version
-  # 2. data.eks_cluster.this.kubernetes_version
-  use_cluster_kubernetes_version  = local.enabled && length(var.kubernetes_version) == 0
-  need_cluster_kubernetes_version = local.use_cluster_kubernetes_version
-
-  resolved_kubernetes_version = local.use_cluster_kubernetes_version ? data.aws_eks_cluster.this[0].version : var.kubernetes_version[0]
+  #   AMI name:      bottlerocket-aws-k8s-1.29-nvidia-x86_64-v1.18.0-7452c37e
+  #   AMI SSM param: /aws/service/bottlerocket/aws-k8s-1.26-nvidia/x86_64/1.18.0/image_id # No "v"
+  #      /aws/service/bottlerocket/aws-k8s-1.26-nvidia/x86_64/1.18.0-7452c37e/image_id
+  # Windows does not allow a specifier for SSM parameters, they only have the latest AMI ID
+  ami_specifier_amazon_linux = {
+    AL2_x86_64             = format("amazon-eks-node-%v-v%v", local.amazon_linux_ami_name_release_part, local.release_version_parts[1])
+    AL2_x86_64_GPU         = format("amazon-eks-gpu-node-%v-v%v", local.amazon_linux_ami_name_release_part, local.release_version_parts[1])
+    AL2_ARM_64             = format("amazon-eks-arm64-node-%v-v%v", local.amazon_linux_ami_name_release_part, local.release_version_parts[1])
+    AL2023_x86_64_STANDARD = format("amazon-eks-node-al2023-x86_64-standard-%v-v%v", local.amazon_linux_ami_name_release_part, local.release_version_parts[1])
+    AL2023_ARM_64_STANDARD = format("amazon-eks-node-al2023-arm64-standard-%v-v%v", local.amazon_linux_ami_name_release_part, local.release_version_parts[1])
+  }
+
+  ami_specifier = length(var.ami_release_version) == 0 ? (local.ami_os == "BOTTLEROCKET" ? "latest" : "recommended") : (
+    lookup(local.ami_specifier_amazon_linux, var.ami_type, var.ami_release_version[0])
+  )
+
+  # As usual, Windows is difficult.
+  is_window_version = local.ami_os == "WINDOWS" && local.ami_specifier != "recommended"
+
+  windows_name_base = {
+    WINDOWS_CORE_2019_x86_64 = "Windows_Server-2019-English-Core-EKS_Optimized"
+    WINDOWS_FULL_2019_x86_64 = "Windows_Server-2019-English-Full-EKS_Optimized"
+    WINDOWS_CORE_2022_x86_64 = "Windows_Server-2022-English-Core-EKS_Optimized"
+    WINDOWS_FULL_2022_x86_64 = "Windows_Server-2022-English-Full-EKS_Optimized"
+  }
+
+  # We do not really need to compute all the names, but it makes debugging easier if we do.
+  ami_name_windows = { for k, v in local.windows_name_base : k => format("%s-%s", v, try(var.ami_release_version[0], "")) }
+
+  fetched_ami_id = try(local.is_window_version ? data.aws_ami.windows_ami[0].image_id : data.aws_ssm_parameter.ami_id[0].insecure_value, "")
+  ami_id         = local.given_ami_id ? var.ami_image_id[0] : local.fetched_ami_id
 }
 
 data "aws_ssm_parameter" "ami_id" {
-  count = local.enabled && local.need_ami_id ? 1 : 0
+  count = local.need_to_get_ami_id && !local.is_window_version ? 1 : 0
 
   name = format(local.ami_ssm_format[var.ami_type], local.ami_specifier, local.resolved_kubernetes_version)
+
+  lifecycle {
+    precondition {
+      condition     = var.ami_type != "CUSTOM"
+      error_message = "The AMI ID must be supplied when AMI type is \"CUSTOM\"."
+    }
+  }
+}
+
+data "aws_ami" "windows_ami" {
+  count = local.need_to_get_ami_id && local.is_window_version ? 1 : 0
+
+  owners = ["amazon"]
+  filter {
+    name   = "name"
+    values = [local.ami_name_windows[var.ami_type]]
+  }
 }
+

docs/terraform.md

@@ -36,6 +36,7 @@
 | [aws_iam_role_policy_attachment.ipv6_eks_cni_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_launch_template.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) | resource |
 | [random_pet.cbd](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
+| [aws_ami.windows_ami](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
 | [aws_eks_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
 | [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.ipv6_eks_cni_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
@@ -50,8 +51,7 @@
 | <a name="input_additional_tag_map"></a> [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.<br>This is for some rare cases where resources want additional configuration of tags<br>and therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no |
 | <a name="input_after_cluster_joining_userdata"></a> [after\_cluster\_joining\_userdata](#input\_after\_cluster\_joining\_userdata) | Additional `bash` commands to execute on each worker node after joining the EKS cluster (after executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production | `list(string)` | `[]` | no |
 | <a name="input_ami_image_id"></a> [ami\_image\_id](#input\_ami\_image\_id) | AMI to use, overriding other AMI specifications, but must match `ami_type`. Ignored if `launch_template_id` is supplied. | `list(string)` | `[]` | no |
-| <a name="input_ami_release_version"></a> [ami\_release\_version](#input\_ami\_release\_version) | OBSOLETE: Use `ami_specifier` instead. Note that it has a different format.<br>Historical description: EKS AMI version to use, e.g. For AL2 \"1.16.13-20200821\" or for bottlerocket \"1.2.0-ccf1b754\" (no \"v\") or  for Windows \"2023.02.14\". For AL2, bottlerocket and Windows, it defaults to latest version for Kubernetes version." | `list(string)` | `[]` | no |
-| <a name="input_ami_specifier"></a> [ami\_specifier](#input\_ami\_specifier) | OS-dependent specifier for one of the several AMIs that match OS, architecture, and Kubernetes 1.xx version.<br>If not specified the recommended/latest AMI for the given Kubernetes version will be used.<br>Unfortunately, the format of this value varies by OS, and we have not found documentation for it.<br>You can generally figure it out from the AMI name or description, and validate it by trying to retrieve<br>the SSM Public Parameter for the AMI ID.<br><br>Examples:<br>  AL2: amazon-eks-node-1.29-v20240117<br>  AL2023: amazon-eks-node-al2023-x86\_64-standard-1.29-v20240605<br>  Bottlerocket: 1.20.1-7c3e9198  \_# Note: 1.20.1 is the Bottlerocket, not Kubernetes, version\_<br>  Windows: <not allowed> | `string` | `"recommended"` | no |
+| <a name="input_ami_release_version"></a> [ami\_release\_version](#input\_ami\_release\_version) | The EKS AMI "release version" to use. Defaults to the latest recommended version.<br>For Amazon Linux, it is the "Release version" from [Amazon AMI Releases](https://github.com/awslabs/amazon-eks-ami/releases)<br>For Bottlerocket, it is the release tag from [Bottlerocket Releases](https://github.com/bottlerocket-os/bottlerocket/releases) without the "v" prefix.<br>For Windows, it is "AMI version" from [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/eks-ami-versions-windows.html).<br>Note that unlike AMI names, release versions never include the "v" prefix.<br>Examples:<br>  AL2: 1.29.3-20240531<br>  Bottlerocket: 1.2.0 or 1.2.0-ccf1b754<br>  Windows: 1.29-2024.04.09 | `list(string)` | `[]` | no |
 | <a name="input_ami_type"></a> [ami\_type](#input\_ami\_type) | Type of Amazon Machine Image (AMI) associated with the EKS Node Group.<br>Defaults to `AL2_x86_64`. Valid values: `AL2_x86_64, AL2_x86_64_GPU, AL2_ARM_64, CUSTOM, BOTTLEROCKET_ARM_64, BOTTLEROCKET_x86_64, BOTTLEROCKET_ARM_64_NVIDIA, BOTTLEROCKET_x86_64_NVIDIA, WINDOWS_CORE_2019_x86_64, WINDOWS_FULL_2019_x86_64, WINDOWS_CORE_2022_x86_64, WINDOWS_FULL_2022_x86_64, AL2023_x86_64_STANDARD, AL2023_ARM_64_STANDARD`. | `string` | `"AL2_x86_64"` | no |
 | <a name="input_associate_cluster_security_group"></a> [associate\_cluster\_security\_group](#input\_associate\_cluster\_security\_group) | When true, associate the default cluster security group to the nodes. If disabled the EKS managed security group will not<br>be associated to the nodes and you will need to provide another security group that allows the nodes to communicate with<br>the EKS control plane. Be aware that if no `associated_security_group_ids` or `ssh_access_security_group_ids` are provided,<br>then the nodes will have no inbound or outbound rules. | `bool` | `true` | no |
 | <a name="input_associated_security_group_ids"></a> [associated\_security\_group\_ids](#input\_associated\_security\_group\_ids) | A list of IDs of Security Groups to associate the node group with, in addition to the EKS' created security group.<br>These security groups will not be modified. | `list(string)` | `[]` | no |
@@ -119,7 +119,6 @@
 
 | Name | Description |
 |------|-------------|
-| <a name="output_WARNING_ami_release_version"></a> [WARNING\_ami\_release\_version](#output\_WARNING\_ami\_release\_version) | Include the warning output message to quite the linter about unused variables. |
 | <a name="output_WARNING_cluster_autoscaler_enabled"></a> [WARNING\_cluster\_autoscaler\_enabled](#output\_WARNING\_cluster\_autoscaler\_enabled) | n/a |
 | <a name="output_eks_node_group_ami_id"></a> [eks\_node\_group\_ami\_id](#output\_eks\_node\_group\_ami\_id) | The ID of the AMI used for the worker nodes, if specified |
 | <a name="output_eks_node_group_arn"></a> [eks\_node\_group\_arn](#output\_eks\_node\_group\_arn) | Amazon Resource Name (ARN) of the EKS Node Group |

examples/complete/fixtures.us-east-2.tfvars

@@ -10,19 +10,28 @@ stage = "test"
 
 name = "eks-node-group"
 
+# Keep Kubernetes version in sync with k8s.io packages in test/src/go.mod
 kubernetes_version = "1.29"
+# Keep the AMI release version in sync with the Kubernetes version
+# Get Release Version from https://github.com/awslabs/amazon-eks-ami/releases
+# but DO NOT USE THE LATEST VERSION. Use the one before that.
+ami_release_version = ["1.29.3-20240531"]
+
+# Use the same architecture for the instance type and the AMI type
+instance_types = ["t4g.small"]
+ami_type       = "AL2023_ARM_64_STANDARD"
+
 
 oidc_provider_enabled = true
 
 enabled_cluster_log_types = ["audit"]
 
 cluster_log_retention_period = 7
 
-instance_types = ["t3.small"]
 
 desired_size = 2
 
-max_size = 3
+max_size = 2
 
 min_size = 2
 
@@ -47,4 +56,3 @@ kubernetes_taints = [
     effect = "PREFER_NO_SCHEDULE"
 }]
 
-ami_type = "AL2023_x86_64_STANDARD"