-
Notifications
You must be signed in to change notification settings - Fork 56
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1 from salesforce/readme
update readme with usage information
- Loading branch information
Showing
1 changed file
with
221 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,225 @@ | ||
### A Genric framework for Writing Mutating Webhook. | ||
# A Generic framework for Writing Mutating Webhook Admission Controllers. | ||
|
||
|
||
|
||
#### How to build & test | ||
## How to build & test | ||
make | ||
|
||
#### Build a docker image | ||
## Build a docker image | ||
make docker | ||
|
||
|
||
## Usage | ||
|
||
### Configuration | ||
|
||
The framework divides the configuration into two parts. | ||
|
||
1. What needs to be injected , aka the sidecars | ||
2. When the injection should be performed, aka the mutation configs | ||
|
||
|
||
The mutating webhook takes the following arguments:- | ||
|
||
``` | ||
/mutating-webhook/mutating-webhook | ||
--sidecar-config-file=/config/sidecarconfig.yaml | ||
--mutation-config-file=/config/mutationconfig.yaml | ||
--cert-file-path=/etc/identity/server/certificates/server.pem | ||
--key-file-path=/etc/identity/server/keys/server-key.pem | ||
``` | ||
|
||
#### Sidecar Configs | ||
|
||
--sidecar-config-file is a list of initcontainers, containers and volumes to inject. The container | ||
and volume yamls are exactly the same as the K8s container and volume format. | ||
|
||
|
||
``` | ||
initContainers: | ||
- name: rsyslog-init | ||
image: blah | ||
command: ["bash", "-c"] | ||
env: | ||
- name: LOG_TYPES_JSON | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: v1 | ||
fieldPath: metadata.annotations['rsyslog.k8s-integration.sfdc.com/log-configs'] | ||
containers: | ||
- name: rsyslog-sidecar | ||
volumes: | ||
- name: rsyslog-spool-vol | ||
``` | ||
|
||
#### Mutation Configs | ||
|
||
--mutation-config-file is a list of mutations that need to be performed by the Mutating Webhook. All the | ||
mutations should belong to the same team. We should not add mutations for things that belong to different | ||
teams. | ||
|
||
Its in the following format:- | ||
|
||
``` | ||
mutationConfigs: | ||
- name: "rsyslog-file-tailer" | ||
annotationNamespace: "rsyslog.k8s-integration.sfdc.com" | ||
annotationTrigger: "inject" | ||
annotationConfig: | ||
volumeMounts: | ||
- name: "volume-mounts" | ||
containerRefs: ["rsyslog-sidecar"] | ||
initContainers: ["rsyslog-init"] | ||
containers: ["rsyslog-sidecar"] | ||
volumes: ["rsyslog-spool-vol", "rsyslog-conf-tpl", "rsyslog-conf-gen"] | ||
volumeMounts: [] | ||
ignoreNamespaces: [] | ||
whitelistNamespaces: [] | ||
``` | ||
|
||
##### Meaning of different terms | ||
|
||
`annotationNamespace`: Every Mutating webhook only looks at annotations within a namespace that it owns. In | ||
the above example, the mutating webhook only looks at the annotations that begin with rsyslog.k8s-integration.sfdc.com | ||
|
||
`annotationTrigger`: The injection is only triggered if the kPod has the following annotation | ||
rsyslog.k8s-integration.sfdc.com/inject present | ||
|
||
|
||
`initContainers`: This is a list of init containers to inject when the annotation is present on | ||
the pod. The name of the initContainers should match an init container in the -sidecar-config-file | ||
|
||
`containers`: This is a list of containers to inject when the annotation is present on the pod. The | ||
name of the containers should match a container in the -sidecar-config-file | ||
|
||
`volumes`: This is a list of volumes to inject when the annotation is present on the pod. The name | ||
of the volumes should match a volume in the -sidecar-config-file | ||
|
||
`annotationConfig`: This is a way of dynamically injecting configuration in the injected containers | ||
which is only known at the pod creation time. Currently it only supports injecting volumeMounts into the injected containers. | ||
|
||
#### Dynamic Configuration of Injected Containers | ||
|
||
``` | ||
annotationConfig: | ||
volumeMounts: | ||
- name: "volume-mounts" | ||
containerRefs: ["rsyslog-sidecar"] | ||
``` | ||
|
||
In the above mutationConfig example, it is instructing the Mutation Webhook to look for an annotation called | ||
volume-mounts and use the value of that annotation to configure a volumeMount inside the container rsyslog-sidecar | ||
which is present in the sidecar configuration. | ||
|
||
The annotation value has configuration for what volume to mount and its mountPath inside the container rsyslog-sidecar. | ||
|
||
Here is the corresponding annotation to expect on the pod. This annotation assumes that the volume “logs” already exists in the Pod. | ||
|
||
|
||
``` | ||
rsyslog.k8s-integration.sfdc.com/volume-mounts: > | ||
[ | ||
{ | ||
"name": "logs", | ||
"mountPath": "/logs" | ||
} | ||
] | ||
``` | ||
|
||
|
||
#### Passing Configuration to Injected Containers as Environment Variables | ||
|
||
Sometimes you need to pass large configuration to the injected containers. One way to do this is by annotations. | ||
|
||
In the Pod, pass an annotation with the required config as follows:- | ||
|
||
``` | ||
rsyslog.k8s-integration.sfdc.com/log-configs: > | ||
[ | ||
{ | ||
"id": "log1", | ||
"source_type": "test:test", | ||
"paths": ["/logs/log1.log"], | ||
"multiline_option": "REGEX", | ||
"start_regex": "^[[:digit:]]{14}\\\\.[[:digit:]]{6}" | ||
} | ||
] | ||
``` | ||
|
||
In the injected container create an environment variable that references this annotation as the source of the value for that environment variable. | ||
|
||
``` | ||
env: | ||
- name: LOG_TYPES_JSON | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: v1 | ||
fieldPath: metadata.annotations['rsyslog.k8s-integration.sfdc.com/log-configs'] | ||
``` | ||
|
||
|
||
#### Example Pod Annotation to that goes with above examples | ||
|
||
``` | ||
# Example pod with rsyslog injection and configuration annotations. | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: rsyslog-inject-example | ||
namespace: test-injection | ||
annotations: | ||
rsyslog.k8s-integration.sfdc.com/inject: enabled | ||
rsyslog.k8s-integration.sfdc.com/volume-mounts: > | ||
[ | ||
{ | ||
"name": "logs", | ||
"mountPath": "/logs" | ||
} | ||
] | ||
rsyslog.k8s-integration.sfdc.com/log-configs: > | ||
[ | ||
{ | ||
"id": "log1", | ||
"source_type": "test:test", | ||
"paths": ["/logs/log1.log"], | ||
} | ||
] | ||
spec: | ||
containers: | ||
- name: app | ||
image: someimage:17 | ||
command: ['sh', '-c', 'while true; do echo -e "20190904013510.766000 [INFO ] log line 1\nline 2\nline 3" >> /logs/log1.log; sleep 10; done'] | ||
volumes: | ||
- name: logs | ||
emptyDir: {} | ||
``` | ||
|
||
|
||
|
||
#### Templating of Sidecar Configuration | ||
|
||
The framework supports golang templating in the sidecar configs. This means certain parts of the injected container | ||
can be derived at runtime from the pod in which the injection needs to happen. | ||
|
||
For e.g. lets say your container has a secret whose name is derived from the service account name of the pod. You | ||
sidecar config can look like this :- | ||
|
||
``` | ||
volumes: | ||
- name: foo | ||
secret: | ||
### This templated field will come from the pod manifest passed to the mutating webhook controller | ||
secretName: aws-iam-{% .Spec.ServiceAccountName %} | ||
``` | ||
|
||
Similarly lets say you want to populate an environment variable in the injected container, where the value of the | ||
environment variable comes from an annotation in the pod. | ||
|
||
``` | ||
- name: VAULT_ROLE | ||
### This templated field will come from the pod manifest passed to the mutating webhook controller | ||
value: {% index .Annotations "vault.k8s-integration.sfdc.com/role" %} | ||
``` | ||
|