From 91100d400c3b8090e71fe498eb7cdfa57163d43e Mon Sep 17 00:00:00 2001 From: Pengfei Ni Date: Thu, 22 Jun 2017 16:38:56 +0800 Subject: [PATCH] Add cri-tools docs --- README.md | 162 ++++----------------------------------------- docs/benchmark.md | 46 +++++++++++++ docs/crictl.md | 124 ++++++++++++++++++++++++++++++++++ docs/roadmap.md | 44 ++++++++++++ docs/validation.md | 48 ++++++++++++++ 5 files changed, 274 insertions(+), 150 deletions(-) create mode 100644 docs/benchmark.md create mode 100644 docs/crictl.md create mode 100644 docs/roadmap.md create mode 100644 docs/validation.md diff --git a/README.md b/README.md index 71dd39595a..96c803fc91 100644 --- a/README.md +++ b/README.md @@ -17,12 +17,15 @@ cri-tools aims to provide a series of debugging and validation tools for Kubelet * Building a new kubelet container runtime based on CRI. * Managing pods/containers for CRI-compatible runtimes by end-users, e.g. pods created by crictl may be removed automatically by kubelet because of non-exist on the kube-apiserver. -## Roadmap +## Current Status -* Basic sandbox/image/container lifecycle operations -* Handle streaming APIs (exec/attach/port-forwarding) -* Reading logs of containers. -* Validation test suites. +We are currently working toward an alpha version of CRI validation tests to be used in conjunction with Kubernetes 1.7. See the [roadmap](docs/roadmap.md) for information about current and future milestones. + +## Documentation + +- [CRI validation test suite](docs/validation.md) +- [CRI performance benchmarking](docs/benchmark.md) +- [CRI CLI](docs/crictl.md) ## Community, discussion, contribution, and support @@ -41,151 +44,10 @@ This is a [Kubernetes Incubator project](https://github.com/kubernetes/community - Champion: Yu-Ju Hong (@yujuhong) - SIG: sig-node -### Code of conduct - -Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md). - -## Getting started - -### Build cri-tools - -```bash -go get github.com/kubernetes-incubator/cri-tools/cmd/critest -go get github.com/kubernetes-incubator/cri-tools/cmd/crictl -``` - -### critest - - Validation test suites for kubelet CRI - -#### Commands - -- `benchmark`,`b`: Benchmark test suite for CRI. - -- `validation`,`v`: Validation test suite for CRI - -#### Global flags - -- `--runtime-endpoint`, `-r`: CRI runtime server endpoint. Default is /var/run/dockershim.sock. - -- `--image-endpoint`, `-i`: CRI server image endpoint, default same as runtime endpoint. - -- `--compile`, `-c`: If we don't need to build dependencies, we can add this flag `--compile=false` or `-c=false`. - -- `--ginkgo-flags`,`-g`: Space-separated list of arguments to pass to Ginkgo test runner. - -- `--focus`,`-f`: CRI test will only run the test that match the focus regular expression. - -- `--skip`,`-s`: CRI test will not run the test that match the focus regular expression. - -### crictl - -CLI for kubelet CRI - -#### Commands - -- `info`: Get version information of runtime +## Contributing -- `sandbox`, `sb`: Manage lifecycle of podsandbox +Interested in contributing? Check out the [documentation](CONTRIBUTING.md). -- `container`, `ctr`: Manage lifecycle of container +## Code of conduct -- `status`: Get the status information of runtime - -- `attach`: Attach a running container - -- `image`: Manage images - -- `exec`: Exec(exec, syncexec) a command in a running container - -- `portforward`: Forword ports(localport:remoteport) from a sandbox - -- `help`, `h`: Shows a list of commands or help for one command - -#### Global flags - -- `--config-file`: Config file in yaml format. Overrided by flags or environment variables. -``` -# cat /etc/crictl.yaml -runtime-endpoint: /var/run/dockershim.sock -image-endpoint: /var/run/dockershim.sock -timeout: 10 -debug: true -``` - -- `--runtime-endpoint`: CRI server runtime endpoint (default: "/var/run/dockershim.sock").The default server is dockershim. If we want to debug other CRI server such as frakti, we can add flag `--runtime-endpoint=/var/run/frakti.sock` - -- `--image-endpoint`: CRI server image endpoint, default same as runtime endpoint. - -- `--timeout`: Timeout of connecting to server (default: 10s) - -- `--debug`: Enable debug output - -- `--help`, `-h`: show help - -- `--version`, `-v`: print the version information of crictl - -#### Examples - -- Run sandbox with config file - -``` -# cat sandbox-config.json -{ - "metadata": { - "name": "nginx-sandbox", - "namespace": "default", - "attempt": 1, - "uid": "hdishd83djaidwnduwk28bcsb" - }, - "linux": { - } -} -# crictl sandbox run sandbox-config.json -9b542bfe8f93eb2d726d0f7b619f253c18858006aa53023e392e138b0be6301c -``` - -- Create container in a sandbox with config file - -``` -# cat sandbox-config.json -{ - "metadata": { - "name": "nginx-sandbox", - "namespace": "default", - "attempt": 1, - "uid": "hdishd83djaidwnduwk28bcsb" - }, - "linux": { - } -} -# cat container-config.json -{ - "metadata": { - "name": "busybox" - }, - "image":{ - "image": "busybox" - }, - "command": [ - "top" - ], - "linux": { - } -} -# crictl container create 9b542bfe8f93eb2d726d0f7b619f253c18858006aa53023e392e138b0be6301c container-config.json sandbox-config.json -bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 -``` - -* Start container - -``` -# crictl container start bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 -bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 -``` - -* Exec a command in container - -``` -# crictl exec -i -t bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 sh -``` +Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md). diff --git a/docs/benchmark.md b/docs/benchmark.md new file mode 100644 index 0000000000..ad85fcdd76 --- /dev/null +++ b/docs/benchmark.md @@ -0,0 +1,46 @@ +# Container Runtime Interface (CRI) Performance Benchmarking + +CRI performance benchmarking provides a benchmarking framework for CRI-compatible container runtimes. This allows the CRI runtime developers to benchmark the performance of their runtime without needing to set up Kubernetes components or run Kubernetes benchmark tests. + +## Install + +The benchmarking tests can be installed easily via `go get` command: + +```sh +go get github.com/kubernetes-incubator/cri-tools/cmd/critest +``` + +Then `critest` binary can be found in `$GOPATH/bin`. + +*Note: ensure GO is installed and GOPATH is set before installing critest.* + +## Running tests + +### Prerequisite + +Before running the test, you need to _ensure that the CRI server under test is running and listening on a Unix socket_. Because the benchmark tests are designed to request changes (e.g., create/delete) to the containers and verify that correct status is reported, it expects to be the only user of the CRI server. Please make sure that 1) there are no existing CRI-managed containers running on the node, and 2) no other processes (e.g., Kubelet) will interfere with the tests. + +### Run + +```sh +critest benchmark +``` + +This will + +- Connect to the shim of CRI container runtime +- Run the benchmark tests using `ginkgo` +- Output the test results to STDOUT + +critest connects to `/var/run/dockershim.sock` by default. For other runtimes, the endpoint can be set in two ways: + +- By setting flags `--runtime-endpoint` and `--image-endpoint` +- By setting environment variables `CRI_RUNTIME_ENDPOINT` and `CRI_IMAGE_ENDPOINT` + +## Additional options + +- `--focus`, `-f`: Only run the tests that match the regular expression. +- -`-ginkgo-flags`, `-g`: Space-separated list of arguments to pass to Ginkgo test runner. +- `--image-endpoint`, `-i`: Set the endpoint of image service. Same with runtime-endpoint if not specified. +- `--runtime-endpoint`, `-r`: Set the endpoint of runtime service. Default to `/var/run/dockershim.sock`. +- `--skip`, `-s`: Skip the tests that match the regular expression. diff --git a/docs/crictl.md b/docs/crictl.md new file mode 100644 index 0000000000..da7364af66 --- /dev/null +++ b/docs/crictl.md @@ -0,0 +1,124 @@ +# Container Runtime Interface (CRI) CLI + +crictl provides a CLI for CRI-compatible container runtimes. This allows the CRI runtime developers to debug of their runtime without needing to set up Kubernetes components. + +crictl is currently Alpha and still under quick iterations. We encourage the CRI developers to report bugs or help extend the coverage by adding more functionalities. + +## Install + +The CRI CLI can be installed easily via `go get` command: + +```sh +go get github.com/kubernetes-incubator/cri-tools/cmd/crictl +``` + +Then `crictl` binary can be found in `$GOPATH/bin`. + +*Note: ensure GO is installed and GOPATH is set before installing crictl.* + +## Usage + +```sh +crictl SUBCOMMAND [FLAGS] +``` + +Subcommands includes: + +- `info`: Get version information of runtime +- `sandbox`, `sb`: Manage lifecycle of podsandbox +- `container`, `ctr`: Manage lifecycle of container +- `status`: Get the status information of runtime +- `attach`: Attach a running container +- `image`: Manage images +- `exec`: Exec(exec, syncexec) a command in a running container +- `portforward`: Forword ports(localport:remoteport) from a sandbox +- `help`, `h`: Shows a list of commands or help for one command + +crictl connects to `/var/run/dockershim.sock` by default. For other runtimes, the endpoint can be set in three ways: + +- By setting flags `--runtime-endpoint` and `--image-endpoint` +- By setting environment variables `CRI_RUNTIME_ENDPOINT` and `CRI_IMAGE_ENDPOINT` +- By setting the endpoint in the config file `--config-file=/etc/crictl.yaml` + +``` +# cat /etc/crictl.yaml +runtime-endpoint: /var/run/dockershim.sock +image-endpoint: /var/run/dockershim.sock +timeout: 10 +debug: true +``` + +## Additional options + +- `--runtime-endpoint`: CRI server runtime endpoint (default: "/var/run/dockershim.sock").The default server is dockershim. If we want to debug other CRI server such as frakti, we can add flag `--runtime-endpoint=/var/run/frakti.sock` +- `--image-endpoint`: CRI server image endpoint, default same as runtime endpoint. +- `--timeout`: Timeout of connecting to server (default: 10s) +- `--debug`: Enable debug output +- `--help`, `-h`: show help +- `--version`, `-v`: print the version information of crictl +- `--config-file`: Config file in yaml format. Overrided by flags or environment variables. + +## Examples + +### Run sandbox with config file + +``` +# cat sandbox-config.json +{ + "metadata": { + "name": "nginx-sandbox", + "namespace": "default", + "attempt": 1, + "uid": "hdishd83djaidwnduwk28bcsb" + }, + "linux": { + } +} +# crictl sandbox run sandbox-config.json +9b542bfe8f93eb2d726d0f7b619f253c18858006aa53023e392e138b0be6301c +``` + +### Create container in a sandbox with config file + +``` +# cat sandbox-config.json +{ + "metadata": { + "name": "nginx-sandbox", + "namespace": "default", + "attempt": 1, + "uid": "hdishd83djaidwnduwk28bcsb" + }, + "linux": { + } +} +# cat container-config.json +{ + "metadata": { + "name": "busybox" + }, + "image":{ + "image": "busybox" + }, + "command": [ + "top" + ], + "linux": { + } +} +# crictl container create 9b542bfe8f93eb2d726d0f7b619f253c18858006aa53023e392e138b0be6301c container-config.json sandbox-config.json +bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 +``` + +### Start container + +``` +# crictl container start bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 +bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 +``` + +### Exec a command in container + +``` +# crictl exec -i -t bf642f55ecf54345354a86a42c08fb0d66e55e90c855973495f31e991c2bf725 sh +``` diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 0000000000..abb3a60116 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,44 @@ +# ROADMAP + +CRI tools is a set of tools for Kubelet Container Runtime Interface (CRI). It is + +- designing for all CRI-compatible container runtimes. +- aiming to help runtime maintainers for validation and debugging +- portable and easy to use + +## Components + +CRI tools is composed of two components: + +- critest: validation and performance benchmark test suites +- crictl: CLI for Kubelet CRI + +## Roadmap + +### Kubernetes v1.7 + +- CRI validation test suites + - Basic sandbox/container operations + - Basic image operations + - networking + - DNS config + - port mapping + - streaming + - exec, attach, portforward + - security context + - hostPID, hostIPC, hostNetwork + - runAsUser, readOnlyRootfs, privileged + - execSync + - version + - status + +### Kubernetes v1.8 + +- Supporting TLS for streaming operations +- Validation test suites for experimental features, e.g. SELinux, Apparmor, Seccomp, Devices, Resource Limits +- Benchmark test suites, e.g. + - Latency + - resource usage (CPU, memory, disk IO, network IO) +- UX improvin, e.g. + - support more output formats: -o json|yaml +- Debugging diff --git a/docs/validation.md b/docs/validation.md new file mode 100644 index 0000000000..68d9698adb --- /dev/null +++ b/docs/validation.md @@ -0,0 +1,48 @@ +# Container Runtime Interface (CRI) Validation Testing + +CRI validation testing provides a test framework and a suite of tests to validate that the Container Runtime Interface (CRI) server implementation meets all the requirements. This allows the CRI runtime developers to verify that their runtime conforms to CRI, without needing to set up Kubernetes components or run Kubernetes end-to-end tests. + +CRI validation testing is currently Alpha and still under quick iterations. We encourage the CRI developers to report bugs or help extend the test coverage by adding more tests. + +## Install + +The test suites can be installed easily via `go get` command: + +```sh +go get github.com/kubernetes-incubator/cri-tools/cmd/critest +``` + +Then `critest` binary can be found in `$GOPATH/bin`. + +*Note: ensure GO is installed and GOPATH is set before installing critest.* + +## Running tests + +### Prerequisite + +Before running the test, you need to _ensure that the CRI server under test is running and listening on a Unix socket_. Because the validation tests are designed to request changes (e.g., create/delete) to the containers and verify that correct status is reported, it expects to be the only user of the CRI server. Please make sure that 1) there are no existing CRI-managed containers running on the node, and 2) no other processes (e.g., Kubelet) will interfere with the tests. + +### Run + +```sh +critest validation +``` + +This will + +- Connect to the shim of CRI container runtime +- Run the tests using `ginkgo` +- Output the test results to STDOUT + +critest connects to `/var/run/dockershim.sock` by default. For other runtimes, the endpoint can be set in two ways: + +- By setting flags `--runtime-endpoint` and `--image-endpoint` +- By setting environment variables `CRI_RUNTIME_ENDPOINT` and `CRI_IMAGE_ENDPOINT` + +## Additional options + +- `--focus`, `-f`: Only run the tests that match the regular expression. +- -`-ginkgo-flags`, `-g`: Space-separated list of arguments to pass to Ginkgo test runner. +- `--image-endpoint`, `-i`: Set the endpoint of image service. Same with runtime-endpoint if not specified. +- `--runtime-endpoint`, `-r`: Set the endpoint of runtime service. Default to `/var/run/dockershim.sock`. +- `--skip`, `-s`: Skip the tests that match the regular expression.