feat: add wiz manifests

[vinaythupili]

-- add wiz manifests

[vinaythupili]

👍

[linki]

We should also populate the deletions.yaml. Otherwise, the ConfigItem cannot be used to turn it off again.

[vinaythupili]

We should also populate the deletions.yaml. Otherwise, the ConfigItem cannot be used to turn it off again.

updated 👍

[mikkeloscar]

please describe with 1-2 lines in a comment what these configs do. See how we do it in the rest of this file.

memory/cpu config-items don't need much description, those are obvious, but the others e.g. like wiz_node_feature_rollout does need a bit of description how to use it.

[mikkeloscar]

If both runtime_sensor and runtime_connector are false, then it should also delete the wiz namespace as a last step.

[mikkeloscar]

please use daemonset: wiz-sensor as the only label selector here. This is how we do it for all other daemonsets/deployments in this repo and it allows easier change of application/component labels if ever needed in the future.

[mikkeloscar]

please use deployment: wiz-connector-agent as the only label selector here. This is how we do it for all other daemonsets/deployments in this repo and it allows easier change of application/component labels if ever needed in the future.

[mikkeloscar]

This should not be node-feature.zalando.org/wiz: enabled or node.kubernetes.io/role: worker. It should always select nodes with node.kubernetes.io/role: worker and optionally it should limit this futher to nodes with: node-feature.zalando.org/wiz: enabled

E.g. you want this:

nodeSelector:
        {{ if eq .Cluster.ConfigItems.wiz_node_feature_rollout "true" }}
        node-feature.zalando.org/wiz: enabled
        {{ end }}
        node.kubernetes.io/role: worker

[mikkeloscar]

This service account should be added to the privileged list here:

kubernetes-on-aws/cluster/manifests/01-admission-control/config.yaml

Lines 95 to 113 in 5f4fdd5

	# service accounts that need privileged PSP should be defined here as `<namespace>_<sa-name>`
	pod.pod-security-policy.privileged-service-accounts.kube-system_kube-proxy: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_skipper-ingress: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_node-monitor: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_nvidia: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_audittrail-adapter: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_kube-aws-iam-controller: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_kube2iam: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_ebs-csi-node-sa: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_flannel: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_etcd-backup: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_coredns: ""
	pod.pod-security-policy.privileged-service-accounts.kube-system_efs-provisioner: ""
	{{- if eq .Cluster.ConfigItems.s3_csi_driver "true" }}
	pod.pod-security-policy.privileged-service-accounts.kube-system_s3-csi-driver: ""
	{{- end }}
	{{- if eq .Cluster.ConfigItems.aws_efa_device_plugin_enabled "true" }}
	pod.pod-security-policy.privileged-service-accounts.kube-system_aws-efa-k8s-device-plugin: ""
	{{- end }}

it should ofc. only be added if eq .Cluster.ConfigItems.wiz_enable_runtime_sensor "true"

This was currently set per cluster via: teapot_admission_controller_pod_security_policy_privileged_service_accounts but it makes more sense to make it a condition of the wiz_enable_runtime_sensor: true|false config-item.

[mikkeloscar]

These capabilities should be enabled conditionally here:

kubernetes-on-aws/cluster/manifests/01-admission-control/config.yaml

Lines 119 to 136 in 5f4fdd5

	pod.pod-security-policy.allowed-restricted-capabilities.AUDIT_WRITE: ""
	pod.pod-security-policy.allowed-restricted-capabilities.CHOWN: ""
	pod.pod-security-policy.allowed-restricted-capabilities.DAC_OVERRIDE: ""
	pod.pod-security-policy.allowed-restricted-capabilities.FOWNER: ""
	pod.pod-security-policy.allowed-restricted-capabilities.FSETID: ""
	pod.pod-security-policy.allowed-restricted-capabilities.KILL: ""
	pod.pod-security-policy.allowed-restricted-capabilities.MKNOD: ""
	pod.pod-security-policy.allowed-restricted-capabilities.NET_BIND_SERVICE: ""
	pod.pod-security-policy.allowed-restricted-capabilities.NET_RAW: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SETFCAP: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SETGID: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SETPCAP: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SETUID: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SYS_CHROOT: ""
	pod.pod-security-policy.allowed-restricted-capabilities.SYS_NICE: ""
	{{- range $cap := split .Cluster.ConfigItems.teapot_admission_controller_pod_security_policy_additional_restricted_capabilities "," }}
	pod.pod-security-policy.allowed-restricted-capabilities.{{ $cap }}: ""
	{{- end}}

(Those that are not already mentioned in the list)

[mikkeloscar]

Note that this memory volume will contribute to the memory limit of the pod. Would be good to understand with wiz if this must be memory backed or could be backed by disk and thereby allow to potentially lower the memory request/limit of daemonset pod.