Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/util/vault.go passes incorrect type for vaultCAVerify #1852

Closed
bnevis-i opened this issue Feb 1, 2021 · 0 comments · Fixed by #1853
Closed

internal/util/vault.go passes incorrect type for vaultCAVerify #1852

bnevis-i opened this issue Feb 1, 2021 · 0 comments · Fixed by #1853
Labels
bug Something isn't working component/util Utility functions shared between CephFS and RBD

Comments

@bnevis-i
Copy link

bnevis-i commented Feb 1, 2021

Describe the bug

The initConnection() method in internal/util/vault.go stores a boolean value in vaultConfig["VAULT_SKIP_VERIFY"] but github.com/libopenstorage/secrets expects configuration values to be strings.

Environment details

  • Image/version of Ceph CSI driver : k8s.gcr.io/sig-storage/csi-provisioner:v2.0.4
  • Helm chart version : none
  • Kernel version : 5.9.12-200.fc33.x86_64
  • Mounter used for mounting PVC (for cephfs its fuse or kernel. for rbd its
    krbd or rbd-nbd) : rbd
  • Kubernetes cluster version : v1.19.2-1008+70708036fc2657-dirty
  • Ceph cluster version : 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable)

Steps to reproduce

Steps to reproduce the behavior:

Create ceph-csi-encryption-kms-config config map with TLS-based Vault and "vaultCAVerify": "false" while using encryption enabled RBD storage pool.

Actual results

Provisioner plugin raises exception that the Vault TLS certificate cannot be verified, regardless of the setting of vaultCAVerify.

Expected behavior

Vault connection is made without supplying custom CA when vaultCAVerify is false.

Logs

csi-provisioner log

I0201 04:08:44.211470       1 controller.go:1317] provision "my-restricted/rbd-fs-user1-pvc" class "csi-rbd-sc": started
I0201 04:08:44.211544       1 controller.go:573] CreateVolumeRequest {Name:pvc-cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b CapacityRange:required_bytes:1073741824  VolumeCapabilities:[mount:<fs_type:"ext4" mount_flags:"discard" > access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[clusterID:1a594345-f0c7-4b31-b415-9daaa82d05b0 csi.storage.k8s.io/controller-expand-secret-name:csi-rbd-secret csi.storage.k8s.io/controller-expand-secret-namespace:ceph-csi csi.storage.k8s.io/node-stage-secret-name:csi-rbd-secret csi.storage.k8s.io/node-stage-secret-namespace:ceph-csi csi.storage.k8s.io/provisioner-secret-name:csi-rbd-secret csi.storage.k8s.io/provisioner-secret-namespace:ceph-csi encrypted:true encryptionKMSID:csi-rbd-sc-vaultconfig encryptionKMSType:vault pool:openshift-pv] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0201 04:08:44.211721       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"my-restricted", Name:"rbd-fs-user1-pvc", UID:"cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b", APIVersion:"v1", ResourceVersion:"1511184", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "my-restricted/rbd-fs-user1-pvc"
I0201 04:08:44.219330       1 connection.go:182] GRPC call: /csi.v1.Controller/CreateVolume
I0201 04:08:44.219355       1 connection.go:183] GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b","parameters":{"clusterID":"1a594345-f0c7-4b31-b415-9daaa82d05b0","csi.storage.k8s.io/pv/name":"pvc-cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b","csi.storage.k8s.io/pvc/name":"rbd-fs-user1-pvc","csi.storage.k8s.io/pvc/namespace":"my-restricted","encrypted":"true","encryptionKMSID":"csi-rbd-sc-vaultconfig","encryptionKMSType":"vault","pool":"openshift-pv"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4","mount_flags":["discard"]}},"access_mode":{"mode":1}}]}
I0201 04:08:44.232461       1 connection.go:185] GRPC response: {}
I0201 04:08:44.232514       1 connection.go:186] GRPC error: rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed creating new Vault Secrets: Put "https://vault.(redacted):8200/v1/auth/kubernetes/login": x509: certificate signed by unknown authority
I0201 04:08:44.232544       1 controller.go:645] CreateVolume failed, supports topology = false, node selected false => may reschedule = false => state = Finished: rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed creating new Vault Secrets: Put "https://vault.(redacted):8200/v1/auth/kubernetes/login": x509: certificate signed by unknown authority
I0201 04:08:44.232593       1 controller.go:1084] Final error received, removing PVC cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b from claims in progress
W0201 04:08:44.232608       1 controller.go:943] Retrying syncing claim "cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b", failure 0
E0201 04:08:44.232631       1 controller.go:966] error syncing claim "cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b": failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed creating new Vault Secrets: Put "https://vault.(redacted):8200/v1/auth/kubernetes/login": x509: certificate signed by unknown authority
I0201 04:08:44.232669       1 controller.go:1317] provision "my-restricted/rbd-fs-user1-pvc" class "csi-rbd-sc": started
I0201 04:08:44.232715       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"my-restricted", Name:"rbd-fs-user1-pvc", UID:"cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b", APIVersion:"v1", ResourceVersion:"1511184", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed creating new Vault Secrets: Put "https://vault.(redacted):8200/v1/auth/kubernetes/login": x509: certificate signed by unknown authority
I0201 04:08:44.232706       1 controller.go:573] CreateVolumeRequest {Name:pvc-cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b CapacityRange:required_bytes:1073741824  VolumeCapabilities:[mount:<fs_type:"ext4" mount_flags:"discard" > access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[clusterID:1a594345-f0c7-4b31-b415-9daaa82d05b0 csi.storage.k8s.io/controller-expand-secret-name:csi-rbd-secret csi.storage.k8s.io/controller-expand-secret-namespace:ceph-csi csi.storage.k8s.io/node-stage-secret-name:csi-rbd-secret csi.storage.k8s.io/node-stage-secret-namespace:ceph-csi csi.storage.k8s.io/provisioner-secret-name:csi-rbd-secret csi.storage.k8s.io/provisioner-secret-namespace:ceph-csi encrypted:true encryptionKMSID:csi-rbd-sc-vaultconfig encryptionKMSType:vault pool:openshift-pv] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0201 04:08:44.232877       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"my-restricted", Name:"rbd-fs-user1-pvc", UID:"cec6ac63-abe5-4f1d-8d55-8f5abee8fa1b", APIVersion:"v1", ResourceVersion:"1511184", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "my-restricted/rbd-fs-user1-pvc"
I0201 04:08:44.235609       1 connection.go:182] GRPC call: /csi.v1.Controller/CreateVolume

Additional context

See related ticket for a fuller description of the issue:
libopenstorage/secrets#51

Suggested change:

Change

vaultConfig[api.EnvVaultInsecure] = !vaultCAVerify

to

vaultConfig[api.EnvVaultInsecure] = strconv.FormatBool(!vaultCAVerify)
@nixpanic nixpanic added bug Something isn't working component/util Utility functions shared between CephFS and RBD labels Feb 2, 2021
nixpanic added a commit to nixpanic/ceph-csi that referenced this issue Feb 2, 2021
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: ceph#1852
Reported-by: Bryon Nevis <[email protected]>
Signed-off-by: Niels de Vos <[email protected]>
@mergify mergify bot closed this as completed in #1853 Feb 2, 2021
mergify bot pushed a commit that referenced this issue Feb 2, 2021
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <[email protected]>
Signed-off-by: Niels de Vos <[email protected]>
mergify bot pushed a commit that referenced this issue Feb 2, 2021
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <[email protected]>
Signed-off-by: Niels de Vos <[email protected]>
(cherry picked from commit 582d004)
mergify bot pushed a commit that referenced this issue Feb 2, 2021
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <[email protected]>
Signed-off-by: Niels de Vos <[email protected]>
(cherry picked from commit 582d004)
nixpanic added a commit to nixpanic/ceph-csi that referenced this issue Feb 2, 2021
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: ceph#1852
Reported-by: Bryon Nevis <[email protected]>
Signed-off-by: Niels de Vos <[email protected]>
(cherry picked from commit 582d004)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component/util Utility functions shared between CephFS and RBD
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants