Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use git sha1 instead of counter tagging #121

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:
# This is needed so that a manifest is created, and we can have the same
# docker container on both x86_64 and arm64.
platforms: linux/amd64,linux/arm64
tags: ${{ env.VERSION }}
tags: ${{ env.VERSION }},${{ env.IMAGE_NAME }}:latest
cache-from: type=gha
cache-to: type=gha,mode=max

Expand Down Expand Up @@ -125,7 +125,7 @@ jobs:
file: Dockerfile.riscv64
push: ${{ github.event_name != 'pull_request' }}
platforms: linux/amd64
tags: ${{ env.VERSION }}-riscv
tags: ${{ env.VERSION }}-riscv,${{ env.IMAGE_NAME }}:latest-riscv
cache-from: type=gha
cache-to: type=gha,mode=max

Expand Down
62 changes: 36 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,15 @@

**`rustvmm/dev`** is a container with all dependencies used for running
`rust-vmm` integration and performance tests. The container is available on
Docker Hub and has support for `x86_64` and `aarch64` platforms.
Docker Hub and has support for `x86_64` `aarch64` and `riscv64` platforms.

For the latest available tag, please check the `rustvmm/dev` builds available
on [Docker Hub](https://hub.docker.com/r/rustvmm/dev/tags).
The latest available tag is `latest` for `x86_64` and `aarch64` and
`latest-riscv` for `riscv64`. If you want `git sha1` lables or previously used
`vN` counter lables, please check the `rustvmm/dev` builds available on
[Docker Hub](https://hub.docker.com/r/rustvmm/dev/tags).

Note: we used counter tagging `vN` for `rustvmm/dev` until `v49`, but now we
switch to `git sha1` tagging. Details are records in #121.

## Know Issues

Expand All @@ -16,21 +21,25 @@ For now rust is installed only for the root user.
The container is currently used for running the integration tests for the
majority of rust-vmm crates.

Example of running cargo build on the kvm-ioctls crate:
Example of running cargo build on the kvm crate:

```bash
> git clone [email protected]:rust-vmm/kvm-ioctls.git
> cd kvm-ioctls/
> docker run --volume $(pwd):/kvm-ioctls \
rustvmm/dev:$VERSION \
/bin/bash -c "cd /kvm-ioctls && cargo build --release"
> git clone https://github.com/rust-vmm/kvm.git
> cd kvm
# latest for x86_64 and aarch64, latest-riscv for riscv64
> docker run --volume $(pwd):/kvm \
rustvmm/dev:latest \
/bin/bash -c "cd /kvm && cargo build --release"
Downloading crates ...
Downloaded libc v0.2.48
Downloaded kvm-bindings v0.1.1
Compiling libc v0.2.48
Compiling kvm-bindings v0.1.1
Compiling kvm-ioctls v0.0.1 (/kvm-ioctls)
Finished release [optimized] target(s) in 5.63s
Downloaded bitflags v1.3.2
Downloaded vmm-sys-util v0.12.1
Compiling libc v0.2.169
Compiling bitflags v1.3.2
Compiling kvm-ioctls v0.20.0 (/kvm/kvm-ioctls)
Compiling bitflags v2.8.0
Compiling vmm-sys-util v0.12.1
Compiling kvm-bindings v0.11.0 (/kvm/kvm-bindings)
Finished `release` profile [optimized] target(s) in 6.34s
```

## Testing Changes locally with the Container Image
Expand All @@ -44,19 +53,20 @@ To do this, first build the rust-vmm container locally by running the commands
> ./docker.sh build
```

since this command will build a new docker image with tag latest version + 1
and will alias it with "latest" tag, when testing the container check the output
of the `./docker.sh build` command and you will see the tag that will be published
with your PR to be sure that the changes introduced by your PR to the CI works
correctly before pusing it upstream.
Example of this output is `Build completed for rustvmm/dev:v38_x86_64`
since this command will build a new docker image with tag `g$(git show -s
--format=%h)` and will alias it with "latest" tag, when testing the container
check the output of the `./docker.sh build` command and you will see the tag
that will be published with your PR to be sure that the changes introduced by
your PR to the CI works correctly before pusing it upstream.
Example of this output is `Build completed for rustvmm/dev:g0c21d2c_x86_64`

Example of how to test the container on your localhost with tag v38_x86_64:
Example of how to test the container on your localhost with tag
`g0c21d2c_x86_64`:

```bash
> docker run --device=/dev/kvm -it --rm \
--volume $(pwd):/path/to/workdir --workdir /path/to/workdir \
--privileged rustvmm/dev:v38_x86_64
--privileged rustvmm/dev:g0c21d2c_x86_64
```
The `--workdir /workdir` option ensures that when the container starts,
the working directory inside the container is set to `/workdir`
Expand Down Expand Up @@ -100,9 +110,9 @@ On an `aarch64` platform:
You will need to redo all steps on an `x86_64` platform so the containers are
kept in sync (same package versions on both `x86_64` and `aarch64`).

Now that the tags `v4_x86_64` and `v4_aarch64` are pushed to Docker Hub, we can
go ahead and also create a new version tag that points to these two builds
using
Now that the tags `g0c21d2c_x86_64` and `g0c21d2c_aarch64` are pushed to Docker
Hub, we can go ahead and also create a new version tag that points to these two
builds using
[docker manifest](https://docs.docker.com/engine/reference/commandline/manifest/).

```bash
Expand Down
29 changes: 8 additions & 21 deletions docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,12 @@ GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
IMAGE_NAME=rustvmm/dev
REGISTRY=index.docker.io

# Get the latest published version. Returns a number.
# If latest is v100, returns 100.
# If latest for riscv64 is v100-riscv, returns 100.
# This works as long as we have less than 100 tags because we set the page size to 100,
# once we have more than that this script needs to be updated.
latest(){
curl -L -s 'https://registry.hub.docker.com/v2/repositories/rustvmm/dev/tags?page_size=100'| \
jq '."results"[]["name"]' | sed 's/"//g' | cut -c 2- | grep -E "^[0-9]+" | sort -n | tail -1
}

next_version() {
latest_version=$(latest)
new_version=$((latest_version + 1))
echo "$new_version"
echo "$(git show -s --format=%h)"
}

print_next_version() {
echo "rustvmm/dev:v$(next_version)"
echo "${IMAGE_NAME}:g$(next_version)"
}

print_registry() {
Expand All @@ -37,14 +25,15 @@ print_image_name() {
# Builds the tag for the newest versions. It needs the last published version number.
# Returns a valid docker tag.
build_tag(){
new_version=$(next_version)
new_tag=${IMAGE_NAME}:v${new_version}_$ARCH
if [ "$ARCH" == "riscv64" ]; then
new_tag=$(print_next_version)-riscv
else
new_tag=$(print_next_version)_$ARCH
fi
echo "$new_tag"
}

# Build a new docker version.
# It will build a new docker image with tag latest version + 1
# and will alias it with "latest" tag.
build(){
new_tag=$(build_tag)
docker build -t "$new_tag" \
Expand All @@ -57,9 +46,7 @@ build(){

# Creates and pushes a manifest for a new version
manifest(){
latest_version=$(latest)
new_version=$((latest_version + 1))
new_tag=${IMAGE_NAME}:v${new_version}
new_tag=$(print_next_version)
docker manifest create \
$new_tag \
"${new_tag}_x86_64" \
Expand Down