How to do "rotate-ca --force" and bring cluster back online

[dante159753]

We are hardening the security of our k3s clusters by enabling them to invalidate all old certs when they are stolen. As currently k3s doesn't support revoke of certs, we are trying to use "rotate-ca --force" to replace all CAs as well as certs.

My cluster is bootstrapped by curl -sfL https://get.k3s.io | K3S_TOKEN=myk3s INSTALL_K3S_SKIP_SELINUX_RPM=true sh -s - server \ --cluster-init
In the document (https://docs.k3s.io/cli/certificate), it mentions I need to rejoin nodes with new secure token and restarts all pods to trust the new CA, but I don't find any secure token under ".env file, systemd unit, or config.yaml" described in the doc. I found secure token in "/var/lib/rancher/k3s/server/token" "/var/lib/rancher/k3s/server/server-token" "/var/lib/rancher/k3s/server/agent", do I need to replace them with new sha256 thumbprint of server-ca? What's the exact steps to complete the force rotation?

[brandond]

As currently k3s doesn't support revoke of certs

Golang itself does not support loading of CRLs. You're really planning on doing a full CA certificate rotation every time you want to revoke access?

What is it specifically you're trying to address? If you're looking at compromise of cluster CA keys, you're best off just destroying the entire cluster, as you can no longer trust anything deployed to it or running on the nodes - as anyone with cluster admin could make changes that would persist even after the certs were rotated.

[dante159753]

We are building product based on k3s, destroy the entire cluster is one way if customers don't care about the workloads. But if customer want to preserve the deployed workloads and data, we want to support full CA rotation to accomplish this. And rotation looks more reasonable as a precaution to handle possible certs exposure.

So is it possible to do a forcefully CA rotation right now?