From 6bc0bb9a23cb1efd95fa524d127c8f8a902dc74c Mon Sep 17 00:00:00 2001 From: Eric Chiang Date: Tue, 27 Jun 2017 15:13:49 -0700 Subject: [PATCH] Update TLS bootstrapping with 1.7 features This includes documenting the new CSR approver built into the controller manager and the kubelet alpha features for certificate rotation. Since the CSR approver changed over the 1.7 release cycle we need to call out the migration steps for those using the alpha feature. This document as a whole could probably use some updates, but the main focus of this PR is just to get these features minimally documented before the release. --- docs/admin/kubelet-tls-bootstrapping.md | 119 ++++++++++++++++++++++-- 1 file changed, 113 insertions(+), 6 deletions(-) diff --git a/docs/admin/kubelet-tls-bootstrapping.md b/docs/admin/kubelet-tls-bootstrapping.md index b3975af1d862e..a918f45279820 100644 --- a/docs/admin/kubelet-tls-bootstrapping.md +++ b/docs/admin/kubelet-tls-bootstrapping.md @@ -1,6 +1,8 @@ --- assignees: +- ericchiang - mikedanese +- jcbsmpsn title: TLS bootstrapping --- @@ -59,18 +61,110 @@ The kube-controller-manager flags are: --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key" ``` -### Automatic approval -To ease deployment and testing, there is an experimental flag in the certificate bootstrapping API to approve all certificate -requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap -token in the token file above. Use of this flag circumvents the approval process described below and is not recommended -for production use. +### Approval controller -The flag is: +In 1.7 the experimenal "group auto approver" controller is dropped in favor of a new `csrapproving` controller +that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default. +The controller uses the [`SubjectAcessReview` API](/docs/admin/authorization/#checking-api-access) to determine +if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent +conflicts with other approvers, the builtin approver doesn't explicitly deny CSRs, only ignoring unauthorized requests. + +The controller categorizes CSRs into three subresources: + +1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`. +2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`. +3. `selfnodeserver` - a node renewing a serving certificate. (ALPHA, requires feature gate) + +The checks to determine if a CSR is a `selfnodeserver` request is currently tied to the kubelet's credential rotation +implementation, an __alpha__ feature. As such, the definition of `selfnodeserver` will likely change in a future and +requires the `RotateKubeletServerCertificate` feature gate on the controller manager. ``` +--feature-gates=RotateKubeletServerCertificate=true +``` + +The following RBAC `ClusterRoles` represent the `nodeclient`, `selfnodeclient`, and `selfnodeserver` capabilities. + +```yml +# A ClusterRole which instructs the CSR approver to approve a user requesting +# node client credentials. +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1beta1 +metadata: + name: approve-node-client-csr +rules: +- apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests/nodeclient"] + verbs: ["create"] +--- +# A ClusterRole which instructs the CSR approver to approve a node renewing its +# own client credentials. +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1beta1 +metadata: + name: approve-node-client-renewal-csr +rules: +- apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests/selfnodeclient"] + verbs: ["create"] +--- +# A ClusterRole which instructs the CSR approver to approve a node requesting a +# serving cert matching its client cert. +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1beta1 +metadata: + name: approve-node-server-renewal-csr +rules: +- apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests/selfnodeserver"] + verbs: ["create"] +``` + +These powers can be granted to credentials, such as bootstrapping tokens. For example, to auto-approve all bootstrapping +tokens initial CSRs, create the following `ClusterRoleBinding`. + +```yml +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1beta1 +metadata: + name: auto-approve-csrs-for-group +subjects: +- kind: Group + name: system:kubelet-bootstrap + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: approve-node-client-csr + apiGroup: rbac.authorization.k8s.io +``` + +This `ClusterRoleBinding` is equivalent to the removed controller manager flag: + +``` +# REMOVED: This flag no longer works. --insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap" ``` +To let a node renew its own credentials, an admin would construct a `ClusterRoleBinding` such as: + +```yml +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1beta1 +metadata: + name: node1-client-cert-renewal +subjects: +- kind: User + name: system:node:node-1 # Let "node-1" renew its client certificate. + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: approve-node-client-renewal-csr + apiGroup: rbac.authorization.k8s.io +``` + +Deleting the binding would prevent the node from renewing its client credentials, and effectively +remove it from the cluster once its certificate expires. + ## kubelet configuration To request a client certificate from kube-apiserver, the kubelet first needs a path to a kubeconfig file that contains the bootstrap authentication token. You can use `kubectl config set-cluster`, `set-credentials`, and `set-context` to build this kubeconfig. Provide the name `kubelet-bootstrap` to `kubectl config set-credentials` and include `--token=` as follows: @@ -87,6 +181,19 @@ The flag to enable this bootstrapping when starting the kubelet is: --experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig" ``` +Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs. +These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature +flags on the kubelet, but may change in backward incompatible ways in future releases. + +``` +--feature-gates=RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true +``` + +`RotateKubeletClientCertificate` causes the kubelet to rotate its client certificates by creating new CSRs as its existing +credentials expire. `RotateKubeletServerCertificate` causes the kubelet to both request a serving certificate after +bootstrapping its client credentials and rotate the certificate. The serving cert currently does not request DNS or IP +SANs. + ## kubectl approval The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an "Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external