Skip to content

Commit

Permalink
Replace tables with bullets
Browse files Browse the repository at this point in the history
Signed-off-by: Anshuman Tripathi <[email protected]>
  • Loading branch information
AnshumanTripathi committed Sep 21, 2024
1 parent 3006e43 commit 82c18fd
Showing 1 changed file with 26 additions and 35 deletions.
61 changes: 26 additions & 35 deletions content/en/docs/concepts/security/hardening-guide/scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,38 +13,35 @@ one of the critical components of the

This document covers how to improve the security posture of the Scheduler.

A misconfigured scheduler can have security implications. Such a scheduler can target specific nodes and evict the workloads or applications that are sharing the node and its resources. This can aid an attacker with a [Yo-Yo attack](https://arxiv.org/abs/2105.00542): an attack on a vulnerable autoscaler.
A misconfigured scheduler can have security implications. Such a scheduler can target specific nodes and evict the workloads or applications that are sharing the node and its resources.
This can aid an attacker with a [Yo-Yo attack](https://arxiv.org/abs/2105.00542): an attack on a vulnerable autoscaler.

<!-- body -->
## kube-scheduler configuration

### Scheduler authentication & authorization command line options
When setting up authentication configuration, it should be made sure that kube-scheduler's authentication remains consistent with kube-api-server's authentication. If any request has missing authentication headers, the authentication should happen through the kube-api-server allowing all authentication to be consistent in the cluster.
{{<table caption="Security advice for kube-scheduler command line options relating to authentication or authorization" >}}
| Command line argument | Security hardening advice |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `authentication-kubeconfig` | Make sure to provide a proper kubeconfig so that the scheduler can retrieve authentication configuration options from the API Server. This kubeconfig file should be protected with strict file permissions. |
| `authentication-tolerate-lookup-failure` | Set this to `false` to make sure the scheduler _always_ looks up its authentication configuration from the API server. |
| `authentication-skip-lookup` | Set this to `false` to make sure the scheduler _always_ looks up its authentication configuration from the API server. |
| `authorization-always-allow-paths` | These paths should respond with data that is appropriate for anonymous authorization. Defaults to `/healthz,/readyz,/livez`. |
| `profiling` | Set to `false` to disable the profiling endpoints which are provide debugging information but which should not be enabled on production clusters as they present a risk of denial of service or information leakage. The `--profiling` argument is deprecated and can now be provided through the [KubeScheduler DebuggingConfiguration](https://kubernetes.io/docs/reference/config-api/kube-scheduler-config.v1/#DebuggingConfiguration). Profiling can be disabled through the kube-scheduler config by setting `enableProfiling` to `false`. |
{{</table>}}


- `authentication-kubeconfig`: Make sure to provide a proper kubeconfig so that the scheduler can retrieve authentication configuration options from the API Server. This kubeconfig file should be protected with strict file permissions.
- `authentication-tolerate-lookup-failure`: Set this to `false` to make sure the scheduler _always_ looks up its authentication configuration from the API server.
- `authentication-skip-lookup`: Set this to `false` to make sure the scheduler _always_ looks up its authentication configuration from the API server.
- `authorization-always-allow-paths`: These paths should respond with data that is appropriate for anonymous authorization. Defaults to `/healthz,/readyz,/livez`.
- `profiling`: Set to `false` to disable the profiling endpoints which are provide debugging information but which should not be enabled on production clusters as they present a risk of denial of service or information leakage. The `--profiling` argument is deprecated and can now be provided through the [KubeScheduler DebuggingConfiguration](https://kubernetes.io/docs/reference/config-api/kube-scheduler-config.v1/#DebuggingConfiguration). Profiling can be disabled through the kube-scheduler config by setting `enableProfiling` to `false`.


### Scheduler networking command line options
{{<table caption="Security advice for kube-scheduler command line options relating to networking" >}}
| Command line argument | Security hardening advice |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `bind-address` | In most cases, the kube-scheduler does not need to be externally accessible. Setting the bind address to `localhost` is a secure practice. |
| `permit-address-sharing` | Set this to `false` to disable connection sharing through `SO_REUSEADDR`. `SO_REUSEADDR` can lead to reuse of terminated connections that are in `TIME_WAIT` state. |
| `permit-port-sharing` | Default `false`. Use the default unless you are confident you understand the security implications. |
{{</table>}}

- `bind-address`: In most cases, the kube-scheduler does not need to be externally accessible. Setting the bind address to `localhost` is a secure practice.
- `permit-address-sharing`: Set this to `false` to disable connection sharing through `SO_REUSEADDR`. `SO_REUSEADDR` can lead to reuse of terminated connections that are in `TIME_WAIT` state.
- `permit-port-sharing`: Default `false`. Use the default unless you are confident you understand the security implications.


### Scheduler TLS command line options
{{<table caption="Security advice for kube-scheduler command line options relating to encryption in transit" >}}
| Command line argument | Security hardening advice |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `requestheader-client-ca-file` | Avoid passing this argument and instead rely on `requestheader-allowed-names` instead. [This configuration makes sure that authentication configuration consistent across the cluster.](https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/#original-request-username-and-group) |
| `tls-cipher-suites` | Always provide a list of preferred cipher suites. This ensures encryption never happens with insecure cipher suites. |
{{</table>}}

- `requestheader-client-ca-file`: Avoid passing this argument and instead rely on `requestheader-allowed-names` instead. [This configuration makes sure that authentication configuration consistent across the cluster.](https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/#original-request-username-and-group)
- `tls-cipher-suites`: Always provide a list of preferred cipher suites. This ensures encryption never happens with insecure cipher suites.


## Scheduling configurations for custom schedulers
This section covers security hardening when using custom schedulers based on the Kubernetes
Expand All @@ -56,11 +53,9 @@ Doing so can affect the defined scheduling behaviors of the kube-scheduler in yo

When using a custom scheduler, plugin extension points such as `queueSort`, `prefilter`, `filter` and `permit` should be used with care.

Exactly one plugin that uses the `queueSort` extension point can be enabled at a time. Any plugins that use `queueSort` should be scrutinized.

Plugins that implement the `prefilter` or `filter` extension point can potentially mark all nodes as unschedulable. This can bring scheduling of new pods to a halt.

Plugins that implement the `permit` extension point can prevent or delay the binding of a Pod. Such plugins should be thoroughly reviewed by the cluster administrator.
- Exactly one plugin that uses the `queueSort` extension point can be enabled at a time. Any plugins that use `queueSort` should be scrutinized.
- Plugins that implement the `prefilter` or `filter` extension point can potentially mark all nodes as unschedulable. This can bring scheduling of new pods to a halt.
- Plugins that implement the `permit` extension point can prevent or delay the binding of a Pod. Such plugins should be thoroughly reviewed by the cluster administrator.

When using a plugin that is not one of the [default plugins](/docs/reference/scheduling/config/#scheduling-plugins), consider disabling the `queueSort`, `filter` and `permit` extension points as follows:

Expand All @@ -81,14 +76,10 @@ profiles:
- name: '*'
```
This creates two scheduler profiles: `default-scheduler` and ` my-custom-scheduler`.
Whenever the `.spec` of a Pod does not have a value for `.spec.schedulerName`,
the kube-scheduler runs for that Pod, using its main configuration, and default plugins.
If you define a Pod with `.spec.schedulerName` set to `my-custom-scheduler`, the
kube-scheduler also runs but with a custom configuration; in that custom configuration,
Whenever the `.spec` of a Pod does not have a value for `.spec.schedulerName`, the kube-scheduler runs for that Pod, using its main configuration, and default plugins.
If you define a Pod with `.spec.schedulerName` set to `my-custom-scheduler`, the kube-scheduler also runs but with a custom configuration; in that custom configuration,
the `queueSort`, `filter` and `permit` extension points are disabled.
If you use this kube-scheduler configuration, and don't run any custom scheduler, and
you then define a Pod with `.spec.schedulerName` set to `nonexistent-scheduler` (or
any other scheduler name that doesn't exist in your cluster), no events would be generated for a pod.
If you use this kube-scheduler configuration, and don't run any custom scheduler, and you then define a Pod with `.spec.schedulerName` set to `nonexistent-scheduler` (or any other scheduler name that doesn't exist in your cluster), no events would be generated for a pod.

## Disallow labeling nodes
A cluster administrator should ensure that cluster users cannot label the nodes. A malicious actor can use `nodeSelector` to schedule workloads on nodes where those workloads should not be present.

0 comments on commit 82c18fd

Please sign in to comment.