From f63a4a8a7a0912f3259716a294624fce97e866c7 Mon Sep 17 00:00:00 2001 From: Sergio Cazzolato Date: Thu, 10 Oct 2024 08:21:07 -0300 Subject: [PATCH] tests: new approach to run tests in ubuntu and debian using snapd deb from the repo (#14496) * tests: new approach to run tests in ubuntu and debian using snapd deb from the repo The idea of this change is to change de default behaviour for the deb pacage used in our spread tests. To simulate a more real scenario, and having snapd snap build from local, this change is adding as default a new scenario to have snapd installed from the deb repository. Then the deb package is saved in the same place as it was built in during the test, making easy to have both scenarios working together. It is a requirement also so be able to run the tests using a built snapd deb as we are currently doing. * Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..33c1d672c6 33c1d672c6 update os.query test 5a428a71ea fix shellcheck 8b423d177a support opensuse 15.6 ab1fa59719 support tests.pkgs download 1e39b307b6 udpate image names for openstack 2a85365d51 remove support for fedora-38 and ubuntu mantic 112bc0a315 append in log-filter 1d4350a818 Make sure the tools are at the begining of the path 45c2f45004 address comments 8743ceaf8c tools: fix support for multi-arch packages on apt-based systems (#54) 33aa8d1e53 fix details in remote.retry test e43b9e314d rename test libraries to avoid static check errors in snapd 4ab8b00afb New spread-filter tool (#53) cdf5cfd47b Remove centos-7 support f3996cc3fa change the spread label 1e309f41c6 change how legacy parameter is determine in remote.pull c43c35f7e3 run remote refresh and wait-for for xenial (skip bionic) 5262d30da7 make sure the test jobs are executed in runners with the spread label cb74259b7a add openstack systems 0b41fd40d3 fix tests.pkgs on arch-linux 558e109793 run fedora-40 spread tests in openstack 6f6187416d fix list implementation b4a5439c9b added more type annotatios to log_helper 58da1e36c3 mypy cleaned 1ff651e680 update wording of remote.pull 18615b1667 just usc scp -O when ssh version is -ge 9 cc68c9868b Added type annotations for log-filter 66f90d10cd Adding -O to scp command to make it compatible in uc24 tests 496cb7b5b3 removing support for centos-8 f2eef30db4 Updated the log helper and log parser 5a375ebf73 Formatting for python utils d3eed3faa5 fix codespell in CODE_OF_CONDUCT.md 18bcca6b14 new log helper d60381fcd9 add run number to filtered filename 5dde2d67b8 consider the tests execution in main 6b9a3aabcc change filtered log name b2756aa579 default file is .filtered.log 500b9dace4 Fix tests workflow 45db26a3d2 fix shellcheck error in log-filter fe45c27b7d create a var to store filter params 5a9b66d7dc filter spread results 51f9b055af New tool used to filter the spread.log output b8d20c1d5b fix snaps.name test with correct siffix spelling f640ac72e3 Add missing test details f0754df304 Filter the error y debug output in log-parser fc10196efd Add suggestions to details 94ac5ffe58 Add details on tests 501578c719 add more checks in os.query to check is-core_xx e8929207ff fix os-query for ubuntu comparing with core 226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems b89ec98b23 use local variables in os.query tool dacfd81de9 fix is_core functions 1db5214d5f Improve the remote docs (#36) 2e4a3153a2 1 more comment 3a0fc57e1e add explanation about why we check for ( Do | Doing ) 4cf8e635bf fix os.query test after merge b89b4f8647 fix artifacts name d30cee6da0 Merge remote-tracking branch 'upstream/main' 5ef5dcbe8f Tests use artifacts in spread tests (#51) 555c43d2ab Support auto-refresh with Do instead of Doing 96c2b0c19c remove tests support for ubuntu 23.04 (EoL) 74082c0c34 Tests improve remote wait (#49) 5121bfb659 remove support for opensuse leap 15.4 (#48) 30df700d08 Add new systems support (#47) 1f08938925 Support check amazon linux version (#46) 43533bdd97 Change the exit value checking for test formats (#45) 3c88244c04 Update check-test-format to support a dir and a list of files (#44) 510d95f429 add extra check for error in auto-refresh detection function 3289d4031b Try open the log with latin-1 encoding when utf-8 is not working 9db785499f improved how the tools are waiting for system reboot 2a5c4414a3 fix shellcheck errors 5e7b63883d Fixes for osquery and tests pkgs (#43) 4c9145e2ac support reboot waiting for auto-refresh 45768f5188 show changes in unknown status after refresh 8013c30c2a Remove support for ubuntu 22.10 b32b80bf54 Fix remote.rait-for test in bionic 5675c625e9 Enable fedora 38 55f4471957 Support for new oss f2e88b357c New tool used to query spread json reports cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42) c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41) ab1e51c29f New comparison in os-query for core systems (#40) e5ae22a5d4 systemd units can be overwritten 63540b845a Fix error messages in remote pull and push 75e8a426a5 make sure the unit is removed in tests.systemd test 9089ff5c02 Update tests to use the new tests.systemd stop-unit 44ecd5e56a Move tests.systemd stop-units to stop-unit 01a2a83b4b Update tests.systemd to have stop units as systemd.sh 162e93bd35 update tests.systemd CLI options to be the same than retry command 14aa43a405 new feature to re-run failed spread tests (#39) 604cb782db Fix shellcheck in systemd tool bfc71082c8 Update the tests.systemd to allow parameters waiting for service status 8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs d90935d2a4 A comment explaining about the default values for wait-for 3232c5dba7 Add support for ubuntu 23.04 a7164fba07 remove fedora 35 support, add fedora 37 support 89b9eb5301 Update systems supported 92bb6a0664 Include snap-sufix in the snaps.name tool git-subtree-dir: tests/lib/external/snapd-testing-tools git-subtree-split: 33c1d672c6512b23892481eeb013bd01e5eb389a * download snapd snap-confine and ubuntu-core-launcher * skip tests that trust on deb pkg testkeys * fix shellcheck * revert kill timeout * fix tests using snapd.tool and store-state test * new tests.info tool this tool is used to check for re-exec and to know the source of the snapd package * skip tests which are testing with re-exec:0 and snap is comming from the repo * update the systems where snapd comes from the repo * skip another test when re-exec:0 * Squashed 'tests/lib/external/snapd-testing-tools/' changes from 33c1d672c6..912131f2a6 912131f2a6 fix another shellcheck d27cc72346 update tests.pkgs to fix shellchecks git-subtree-dir: tests/lib/external/snapd-testing-tools git-subtree-split: 912131f2a6a7d3c807c890d94fe56367aca67039 * Adjust failing test to new re-exec scenario * fix other 2 tests using correct paths * Squashed 'tests/lib/external/snapd-testing-tools/' changes from 912131f2a6..bae7faf8cb bae7faf8cb Update the quoting used to download packages git-subtree-dir: tests/lib/external/snapd-testing-tools git-subtree-split: bae7faf8cb3f04296bac9552ac0d1aa89d4e9139 * some tests fixes Changes considering the changes done in the re-exec workflow * New set of test fixes Also we are inclugin some minor changes to address review comments * update tests info command to check re-execution * fix uc16 which preparing * check SPREAD_SYSTEM is not core in tests.info This is needed because the check in prepare-restore is done before we have an ubuntu-core image * use .skip in tests to determine if a phase has to be skipped Use snap-mount-dir to determine where snaps are mounted * skip updating core snap when the snap pkg is not built from local * save installed core snap when snap deb is used from the repo * copy also .orig core snap * change how we copy the core snap * change the snapname used for core snap * skip saving the core snap * Fix error using snapcraft for testing in arm * make sure fips tests dont run with reexec=0 and snapd.deb from the repo * simplify how fips tests are skipped * fix indentation on test * Make sure we save core snap in TESTSTMP/core_snap * use tests.exec tool to skip tests * updated failing tests ude to new re-execution mechanism * fix shellcheck on test preseed-lxd * Fix tests Fix tests that needed to be updated after snapd snap is build from current * fix how the current snapd snap is found * updated the name of the command is-snapd-pkg-repo --- spread.yaml | 8 +++ tests/bin/tests.exec | 1 + tests/bin/tests.info | 1 + tests/lib/prepare-restore.sh | 7 ++- tests/lib/prepare.sh | 63 +++++++++++++------ tests/lib/tools/snapd.tool | 11 +++- tests/lib/tools/tests.exec | 58 +++++++++++++++++ tests/lib/tools/tests.info | 62 ++++++++++++++++++ .../main/classic-custom-device-reg/task.yaml | 17 ++--- tests/main/classic-firstboot/task.yaml | 16 ++--- .../classic-prepare-image-no-core/task.yaml | 17 ++--- tests/main/classic-prepare-image/task.yaml | 17 ++--- tests/main/classic-snapd-firstboot/task.yaml | 16 ++--- tests/main/command-chain/task.yaml | 6 ++ tests/main/debug-execution/task.yaml | 5 ++ tests/main/fips/task.yaml | 9 +++ tests/main/install-errors/task.yaml | 6 ++ tests/main/install-sideload-epochs/task.yaml | 5 ++ tests/main/install-sideload/task.yaml | 6 ++ .../main/interfaces-personal-files/task.yaml | 4 ++ tests/main/known/task.yaml | 2 + tests/main/postrm-purge/task.yaml | 12 ++++ tests/main/preseed-lxd/task.yaml | 46 ++++++++++++-- tests/main/preseed/task.yaml | 15 ++++- tests/main/searching/task.yaml | 2 + .../security-device-cgroups-helper/task.yaml | 7 +++ tests/main/snap-confine-tmp-mount/task.yaml | 4 ++ tests/main/snap-run-devmode-classic/task.yaml | 45 ++++--------- tests/main/snap-run-hook/task.yaml | 5 ++ tests/main/snap-run/task.yaml | 6 ++ tests/main/snap-seccomp/task.yaml | 5 ++ tests/main/snapd-homedirs-vendored/task.yaml | 13 +++- tests/main/snapd-reexec/task.yaml | 38 ++++++++--- .../task.yaml | 8 ++- tests/main/snapd-snap/task.yaml | 7 ++- tests/main/snapd-update-services/task.yaml | 13 ++-- tests/main/store-state/task.yaml | 18 +++--- tests/main/try/task.yaml | 6 ++ tests/main/upgrade-from-release/task.yaml | 2 +- tests/main/user-session-env/task.yaml | 9 ++- tests/main/writable-areas/task.yaml | 6 ++ tests/regression/lp-1871652/task.yaml | 3 +- tests/smoke/find-info/task.yaml | 3 +- tests/smoke/install/task.yaml | 6 ++ tests/smoke/remove/task.yaml | 2 + tests/smoke/sandbox/task.yaml | 4 ++ tests/smoke/versioning/task.yaml | 2 + 47 files changed, 467 insertions(+), 157 deletions(-) create mode 120000 tests/bin/tests.exec create mode 120000 tests/bin/tests.info create mode 100755 tests/lib/tools/tests.exec create mode 100755 tests/lib/tools/tests.info diff --git a/spread.yaml b/spread.yaml index 53f02a6162c..b50d3de173f 100644 --- a/spread.yaml +++ b/spread.yaml @@ -82,8 +82,12 @@ environment: SNAPD_NO_MEMORY_LIMIT: '$(HOST: echo "${SPREAD_SNAPD_NO_MEMORY_LIMIT:-}")' SNAPD_PUBLISHED_VERSION: '$(HOST: echo "$SPREAD_SNAPD_PUBLISHED_VERSION")' + # Use the snapd package from the repository when possible + SNAPD_DEB_FROM_REPO: '$(HOST: echo "${SPREAD_SNAPD_DEB_FROM_REPO:-true}")' # Build and use snapd from current branch BUILD_SNAPD_FROM_CURRENT: '$(HOST: echo "${SPREAD_BUILD_SNAPD_FROM_CURRENT:-true}")' + # Directory where the built snapd snaps and other assets are stored + SNAPD_WORK_DIR: '$(HOST: echo "${SPREAD_SNAPD_WORK_DIR:-/tmp/work-dir}")' # Directory where the nested images and test assets are stored NESTED_WORK_DIR: '$(HOST: echo "${NESTED_WORK_DIR:-/tmp/work-dir}")' @@ -1430,6 +1434,10 @@ suites: "$TESTSLIB"/prepare-restore.sh --prepare-suite prepare-each: | "$TESTSLIB"/prepare-restore.sh --prepare-suite-each + # When snapd.deb is from the repo, we just need to test when re-exec is enabled + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "When snapd.deb is from the repo, we just need to test when re-exec is enabled" && exit 0 + fi restore-each: | "$TESTSLIB"/prepare-restore.sh --restore-suite-each restore: | diff --git a/tests/bin/tests.exec b/tests/bin/tests.exec new file mode 120000 index 00000000000..689b4eb7189 --- /dev/null +++ b/tests/bin/tests.exec @@ -0,0 +1 @@ +../lib/tools/tests.exec \ No newline at end of file diff --git a/tests/bin/tests.info b/tests/bin/tests.info new file mode 120000 index 00000000000..247c842a58c --- /dev/null +++ b/tests/bin/tests.info @@ -0,0 +1 @@ +../lib/tools/tests.info \ No newline at end of file diff --git a/tests/lib/prepare-restore.sh b/tests/lib/prepare-restore.sh index bd673de9845..5a0196dd7a8 100755 --- a/tests/lib/prepare-restore.sh +++ b/tests/lib/prepare-restore.sh @@ -587,7 +587,12 @@ prepare_project() { # go mod runs as root and will leave strange permissions chown test:test -R "$SPREAD_PATH" - if [ "$BUILD_SNAPD_FROM_CURRENT" = true ]; then + # We are testing snapd snap on top of snapd from the archive + # of the tested distribution. Download snapd and snap-confine + # as they exist in the archive for further use. + if tests.info is-snapd-from-archive; then + ( cd "${GOHOME}" && tests.pkgs download snapd snap-confine) + elif [ "$BUILD_SNAPD_FROM_CURRENT" = true ]; then case "$SPREAD_SYSTEM" in ubuntu-*|debian-*) build_deb diff --git a/tests/lib/prepare.sh b/tests/lib/prepare.sh index 10605402580..1df27bb208e 100755 --- a/tests/lib/prepare.sh +++ b/tests/lib/prepare.sh @@ -9,8 +9,6 @@ set -eux # shellcheck source=tests/lib/state.sh . "$TESTSLIB/state.sh" -: "${WORK_DIR:=/tmp/work-dir}" - disable_kernel_rate_limiting() { # kernel rate limiting hinders debugging security policy so turn it off echo "Turning off kernel rate-limiting" @@ -154,6 +152,24 @@ setup_experimental_features() { fi } +save_installed_core_snap() { + local target_dir="${1-}" + + SNAP_MOUNT_DIR="$(os.paths snap-mount-dir)" + core="$(readlink -f "$SNAP_MOUNT_DIR"/core/current)" + snap="$(mount | awk -v core="$core" '{ if ($3 == core) print $1 }' | head -n1)" + snap_name="$(basename "$snap")" + + # make a copy for later use + if [ -n "$target_dir" ]; then + mkdir -p "$target_dir" + + cp -av "$snap" "${target_dir}/${snap_name}" + cp "$snap" "${target_dir}/${snap_name}.orig" + fi +} + + # update_core_snap_for_classic_reexec modifies the core snap for snapd re-exec # by injecting binaries from the installed snapd deb built from our modified code. # $1: directory where updated core snap should be copied (optional) @@ -175,8 +191,8 @@ update_core_snap_for_classic_reexec() { LIBEXEC_DIR="$(os.paths libexec-dir)" # First of all, unmount the core - core="$(readlink -f "$SNAP_MOUNT_DIR/core/current" || readlink -f "$SNAP_MOUNT_DIR/ubuntu-core/current")" - snap="$(mount | grep " $core" | head -n 1 | awk '{print $1}')" + core="$(readlink -f "$SNAP_MOUNT_DIR"/core/current)" + snap="$(mount | awk -v core="$core" '{ if ($3 == core) print $1 }' | head -n1)" umount --verbose "$core" # Now unpack the core, inject the new snap-exec/snapctl into it @@ -401,15 +417,16 @@ prepare_classic() { # This also prevents snapd from automatically installing snapd snap as # prerequisite for installing any non-base snap introduced in PR#14173. if snap list snapd ; then - snap info snapd - echo "Error: not expecting snapd snap to be installed" - exit 1 + snap info snapd + echo "Error: not expecting snapd snap to be installed" + exit 1 else - build_dir="$WORK_DIR/snapd_snap_for_classic" - rm -rf "$build_dir" - mkdir -p "$build_dir" - build_snapd_snap "$build_dir" - snap install --dangerous "$build_dir/"snapd_*.snap + build_dir="$SNAPD_WORK_DIR/snapd_snap_for_classic" + rm -rf "$build_dir" + mkdir -p "$build_dir" + build_snapd_snap "$build_dir" + snap install --dangerous "$build_dir/"snapd_*.snap + snap wait system seed.loaded fi snap list snapd @@ -450,10 +467,17 @@ prepare_classic() { snap list | grep core - systemctl stop snapd.{service,socket} - # repack and also make a side copy of the core snap - update_core_snap_for_classic_reexec "$TESTSTMP/core_snap" - systemctl start snapd.{service,socket} + # With reexec, and on classic, the snapd snap is preferred over the core snap for reexecution target, + # so to be as close as possible to the actual real life scenarios, we only update the snapd snap. + # The tests alreday ensure that snapd snap is installed. + if tests.info is-snapd-from-archive; then + save_installed_core_snap "$TESTSTMP/core_snap" + else + systemctl stop snapd.{service,socket} + # repack and also make a side copy of the core snap + update_core_snap_for_classic_reexec "$TESTSTMP/core_snap" + systemctl start snapd.{service,socket} + fi prepare_reexec_override prepare_memory_limit_override @@ -523,7 +547,7 @@ build_snapd_snap() { local snapd_snap_cache TARGET="${1}" - snapd_snap_cache="$WORK_DIR/snapd_snap" + snapd_snap_cache="$SNAPD_WORK_DIR/snapd_snap" mkdir -p "${snapd_snap_cache}" for snap in "${snapd_snap_cache}"/snapd_*.snap; do if ! [ -f "${snap}" ]; then @@ -547,6 +571,7 @@ build_snapd_snap() { ;; esac [ -d "${TARGET}" ] || mkdir -p "${TARGET}" + touch "${PROJECT_PATH}"/test-build chmod -R go+r "${PROJECT_PATH}/tests" # TODO: run_snapcraft does not currently guarantee or check the required version for building snapd run_snapcraft --use-lxd --verbosity quiet --output="snapd_from_snapcraft.snap" @@ -565,7 +590,7 @@ build_snapd_snap_with_run_mode_firstboot_tweaks() { TARGET="${1}" - snapd_snap_cache="$WORK_DIR/snapd_snap_with_tweaks" + snapd_snap_cache="$SNAPD_WORK_DIR/snapd_snap_with_tweaks" mkdir -p "${snapd_snap_cache}" for snap in "${snapd_snap_cache}"/snapd_*.snap; do if [ -f "${snap}" ]; then @@ -1192,7 +1217,7 @@ setup_reflash_magic() { # FIXME: install would be better but we don't have dpkg on # the image # unpack our freshly build snapd into the new snapd snap - dpkg-deb -x "$SPREAD_PATH"/../snapd_*.deb "$UNPACK_DIR" + dpkg-deb -x "$GOHOME"/snapd_*.deb "$UNPACK_DIR" # Debian packages don't carry permissions correctly and we use # post-inst hooks to fix that on classic systems. Here, as a special # case, fix the void directory we just unpacked. diff --git a/tests/lib/tools/snapd.tool b/tests/lib/tools/snapd.tool index 980a514125b..6af476b5e1e 100755 --- a/tests/lib/tools/snapd.tool +++ b/tests/lib/tools/snapd.tool @@ -47,8 +47,15 @@ main() { esac done - LIBEXEC_DIR="$(os.paths libexec-dir)" - exec "$LIBEXEC_DIR/snapd/$tool" "$@" + # Use the tool from the snapd snap when re-exec is enabled + # Otherwise use the tool from the distro installation + if tests.info is-reexec-in-use; then + MOUNT_DIR="$(os.paths snap-mount-dir)" + exec "$MOUNT_DIR"/snapd/current/usr/lib/snapd/"$tool" "$@" + else + LIBEXEC_DIR="$(os.paths libexec-dir)" + exec "$LIBEXEC_DIR/snapd/$tool" "$@" + fi } main "$@" \ No newline at end of file diff --git a/tests/lib/tools/tests.exec b/tests/lib/tools/tests.exec new file mode 100755 index 00000000000..d33ae520bbc --- /dev/null +++ b/tests/lib/tools/tests.exec @@ -0,0 +1,58 @@ +#!/bin/bash + +set -eu + +show_help() { + echo "usage: tests.exec skip-test [MSG]" + echo "usage: tests.exec is-skipped" + echo + echo "Supported commands:" + echo " skip-test: indicates the test has to be skipped and saves the raeson" + echo " is-skipped: check if the test has to be skipped and prints the raeson" +} + +skip_test() { + local raeson="${1:-}" + echo "$raeson" > tests.exec +} + +is_skipped() { + if [ -f tests.exec ]; then + echo "skip raeson: $(cat tests.exec)" + return 0 + fi + return 1 +} + +main() { + if [ $# -eq 0 ]; then + show_help + exit 0 + fi + + local subcommand="$1" + local action= + while [ $# -gt 0 ]; do + case "$subcommand" in + -h|--help) + show_help + exit 0 + ;; + *) + action=$(echo "$subcommand" | tr '-' '_') + shift + break + ;; + esac + done + + if [ -z "$(declare -f "$action")" ]; then + echo "tests.exec: no such command: $subcommand" + show_help + exit 1 + fi + + "$action" "$@" +} + +main "$@" diff --git a/tests/lib/tools/tests.info b/tests/lib/tools/tests.info new file mode 100755 index 00000000000..2fe954f747c --- /dev/null +++ b/tests/lib/tools/tests.info @@ -0,0 +1,62 @@ +#!/bin/bash + +set -eu + +show_help() { + echo "usage: tests.info " + echo + echo "Supported commands:" + echo " is-snapd-from-archive: indicates when the snapd package is from the repository" + echo " is-reexec-enabled: indicates re-execution has been explicitly enabled through the env" + echo " is-reexec-in-use: indicates re-execution to the snapd snap is being effectively performed" +} + +# ubuntu 14.04: does not support apt download which is used to get the packages from the archive. +# ubuntu-core: it is built based on the snapd.deb file, so it requires the deb is created from current. +is_snapd_from_archive() { + [ "$SNAPD_DEB_FROM_REPO" = true ] && \ + [ -n "$(command -v apt)" ] && \ + [[ "$SPREAD_SYSTEM" != ubuntu-core-* ]] && \ + not os.query is-trusty +} + +is_reexec_enabled() { + snap debug execution snap | grep -q "is-reexec-enabled: true" +} + +is_reexec_in_use() { + snap debug execution snap | grep -q "is-reexecd: true" +} + +main() { + if [ $# -eq 0 ]; then + show_help + exit 0 + fi + + local subcommand="$1" + local action= + while [ $# -gt 0 ]; do + case "$subcommand" in + -h|--help) + show_help + exit 0 + ;; + *) + action=$(echo "$subcommand" | tr '-' '_') + shift + break + ;; + esac + done + + if [ -z "$(declare -f "$action")" ]; then + echo "tests.info: no such command: $subcommand" + show_help + exit 1 + fi + + "$action" "$@" +} + +main "$@" diff --git a/tests/main/classic-custom-device-reg/task.yaml b/tests/main/classic-custom-device-reg/task.yaml index f263cc6c3c2..a82f1e0e97b 100644 --- a/tests/main/classic-custom-device-reg/task.yaml +++ b/tests/main/classic-custom-device-reg/task.yaml @@ -15,9 +15,9 @@ environment: SEED_DIR: /var/lib/snapd/seed prepare: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ] || tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 fi snap pack "$TESTSLIB/snaps/classic-gadget" @@ -52,20 +52,15 @@ prepare: | systemd-run --unit fakedevicesvc fakedevicesvc localhost:11029 restore: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 + systemctl stop snapd.service snapd.socket fakedevicesvc rm -rf "$SEED_DIR" systemctl start snapd.socket snapd.service execute: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 #shellcheck source=tests/lib/core-config.sh . "$TESTSLIB"/core-config.sh diff --git a/tests/main/classic-firstboot/task.yaml b/tests/main/classic-firstboot/task.yaml index 1a1d6373d44..187a71ad95a 100644 --- a/tests/main/classic-firstboot/task.yaml +++ b/tests/main/classic-firstboot/task.yaml @@ -14,9 +14,9 @@ environment: SEED_DIR: /var/lib/snapd/seed prepare: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ] || tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 fi snap pack "$TESTSLIB/snaps/basic" @@ -51,19 +51,13 @@ prepare: | cp ./test-snapd-service_1.0_all.snap "$SEED_DIR/snaps/test-snapd-service.snap" restore: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 rm -rf "$SEED_DIR" systemctl start snapd.socket snapd.service execute: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 echo "Start the daemon with an empty state, this will make it import " echo "assertions from the $SEED_DIR/assertions subdirectory and " diff --git a/tests/main/classic-prepare-image-no-core/task.yaml b/tests/main/classic-prepare-image-no-core/task.yaml index 8c73e03841b..d52c45b3492 100644 --- a/tests/main/classic-prepare-image-no-core/task.yaml +++ b/tests/main/classic-prepare-image-no-core/task.yaml @@ -20,9 +20,9 @@ environment: SEED_DIR: /var/lib/snapd/seed prepare: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ] || tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 fi "$TESTSTOOLS"/store-state setup-fake-store "$STORE_DIR" @@ -49,10 +49,8 @@ prepare: | systemd-run --unit fakedevicesvc fakedevicesvc localhost:11029 restore: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 + systemctl stop snapd.service snapd.socket fakedevicesvc rm -rf "$SEED_DIR" @@ -61,10 +59,7 @@ restore: | rm -rf "$ROOT" execute: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 #shellcheck source=tests/lib/core-config.sh . "$TESTSLIB"/core-config.sh diff --git a/tests/main/classic-prepare-image/task.yaml b/tests/main/classic-prepare-image/task.yaml index 52e3130e065..c2d755b5df8 100644 --- a/tests/main/classic-prepare-image/task.yaml +++ b/tests/main/classic-prepare-image/task.yaml @@ -19,9 +19,9 @@ environment: SEED_DIR: /var/lib/snapd/seed prepare: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ] || tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 fi "$TESTSTOOLS"/store-state setup-fake-store "$STORE_DIR" @@ -48,10 +48,8 @@ prepare: | systemd-run --unit fakedevicesvc fakedevicesvc localhost:11029 restore: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 + systemctl stop snapd.service snapd.socket fakedevicesvc rm -rf "$SEED_DIR" @@ -60,10 +58,7 @@ restore: | rm -rf "$ROOT" execute: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 #shellcheck source=tests/lib/core-config.sh . "$TESTSLIB"/core-config.sh diff --git a/tests/main/classic-snapd-firstboot/task.yaml b/tests/main/classic-snapd-firstboot/task.yaml index b3631828bc8..98c7da85a11 100644 --- a/tests/main/classic-snapd-firstboot/task.yaml +++ b/tests/main/classic-snapd-firstboot/task.yaml @@ -12,9 +12,9 @@ environment: SEED_DIR: /var/lib/snapd/seed prepare: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ] || tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 fi snap pack "$TESTSLIB/snaps/basic18" @@ -51,19 +51,13 @@ prepare: | cp ./basic18_1.0_all.snap "$SEED_DIR/snaps/basic18.snap" restore: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 rm -rf "$SEED_DIR" systemctl start snapd.socket snapd.service execute: | - if [ "$TRUST_TEST_KEYS" = "false" ]; then - echo "This test needs test keys to be trusted" - exit - fi + tests.exec is-skipped && exit 0 echo "Start the daemon with an empty state, this will make it import " echo "assertions from the $SEED_DIR/assertions subdirectory and " diff --git a/tests/main/command-chain/task.yaml b/tests/main/command-chain/task.yaml index 7ad5bea5e66..e8bbf79d92d 100644 --- a/tests/main/command-chain/task.yaml +++ b/tests/main/command-chain/task.yaml @@ -16,6 +16,10 @@ environment: ENVDUMP: /var/snap/command-chain/current/env prepare: | + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi + echo "Build command chain snap" snap pack command-chain snap install --dangerous command-chain_1.0_all.snap @@ -28,6 +32,8 @@ prepare: | fi execute: | + tests.exec is-skipped && exit 0 + echo "Test that command-chain runs for hooks" [ "$(cat "$BREADCRUMB")" = "chain1 chain2 configure" ] MATCH '^CHAIN_1_RAN=1$' < "$ENVDUMP" diff --git a/tests/main/debug-execution/task.yaml b/tests/main/debug-execution/task.yaml index 064da5df086..afabab74127 100644 --- a/tests/main/debug-execution/task.yaml +++ b/tests/main/debug-execution/task.yaml @@ -8,6 +8,11 @@ debug: | grep -n '' snap-*.out || true execute: | + # TODO: remove this check once snapd 2.65 is released + if tests.info is-snapd-from-archive; then + exit 0 + fi + snap debug execution snap > snap-default.out SNAP_REEXEC=0 snap debug execution snap > snap-no-reexec.out SNAP_REEXEC=1 snap debug execution snap > snap-yes-reexec.out diff --git a/tests/main/fips/task.yaml b/tests/main/fips/task.yaml index 3b0bf025066..e3e1734acb9 100644 --- a/tests/main/fips/task.yaml +++ b/tests/main/fips/task.yaml @@ -7,10 +7,19 @@ systems: - ubuntu-fips-* debug: | + tests.exec is-skipped && exit 0 + cat snapd-map-fips.out || true cat snapd-map-non-fips.out || true execute: | + tests.exec is-skipped && exit 0 + + # In this scenario, the keys from the snapd pkg are used + if [ "$TRUST_TEST_KEYS" = "false" ]; then + tests.exec skip-test "This test needs test keys to be trusted" && exit 0 + fi + pmap -p "$(pidof snapd)" > snapd-map.out case "$SPREAD_SYSTEM" in ubuntu-fips-*) diff --git a/tests/main/install-errors/task.yaml b/tests/main/install-errors/task.yaml index b219856980b..e582e7cf177 100644 --- a/tests/main/install-errors/task.yaml +++ b/tests/main/install-errors/task.yaml @@ -14,10 +14,16 @@ environment: SNAP_REEXEC/withreexec: 1 prepare: | + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi + echo "Given a snap with a failing command is installed" "$TESTSTOOLS"/snaps-state install-local "$SNAP_NAME" execute: | + tests.exec is-skipped && exit 0 + echo "Install unexisting snap prints error" if snap install unexisting.canonical; then echo "Installing unexisting snap should fail" diff --git a/tests/main/install-sideload-epochs/task.yaml b/tests/main/install-sideload-epochs/task.yaml index c70c2dda5c7..446f4d728f6 100644 --- a/tests/main/install-sideload-epochs/task.yaml +++ b/tests/main/install-sideload-epochs/task.yaml @@ -12,10 +12,15 @@ environment: SNAP_REEXEC/reexec1: 1 prepare: | + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi snap pack test-snapd-epoch-1 snap pack test-snapd-epoch-2 execute: | + tests.exec is-skipped && exit 0 + rx="cannot refresh \"[^ \"]*\" to local snap with epoch [^ ]*, because it can't read the current epoch" snap try test-snapd-epoch-1 not snap try test-snapd-epoch-2 2> try.err diff --git a/tests/main/install-sideload/task.yaml b/tests/main/install-sideload/task.yaml index 506cf59f81e..227ed20d674 100644 --- a/tests/main/install-sideload/task.yaml +++ b/tests/main/install-sideload/task.yaml @@ -14,6 +14,10 @@ environment: SNAP_REEXEC/reexec1: 1 prepare: | + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi + for snap in basic test-snapd-tools basic-desktop test-snapd-devmode snap-hooks-bad-install; do snap pack "$TESTSLIB"/snaps/$snap done @@ -26,6 +30,8 @@ prepare: | fi execute: | + tests.exec is-skipped && exit 0 + echo "Sideloaded snap shows status" expected='^basic 1.0 installed$' snap install --dangerous ./basic_1.0_all.snap | MATCH "$expected" diff --git a/tests/main/interfaces-personal-files/task.yaml b/tests/main/interfaces-personal-files/task.yaml index c48839e742c..11e75684364 100644 --- a/tests/main/interfaces-personal-files/task.yaml +++ b/tests/main/interfaces-personal-files/task.yaml @@ -86,6 +86,10 @@ execute: | echo "Find snap-confine" SNAP_CONFINE=$(os.paths libexec-dir)/snapd/snap-confine + if tests.info is-snapd-from-archive; then + MOUNT_DIR="$(os.paths snap-mount-dir)" + SNAP_CONFINE="$MOUNT_DIR/snapd/current$SNAP_CONFINE" + fi # on Ubuntu Core we need to use the correct path to ensure it is # appropriately confined by apparmor as it may be from the snapd # snap diff --git a/tests/main/known/task.yaml b/tests/main/known/task.yaml index 4a1927f1cd5..8440173a77e 100644 --- a/tests/main/known/task.yaml +++ b/tests/main/known/task.yaml @@ -7,6 +7,8 @@ details: | properties in general. execute: | + tests.exec is-skipped && exit 0 + echo "Listing all account assertions" snap known account|MATCH "^type: account$" snap known account|MATCH "^account-id: canonical$" diff --git a/tests/main/postrm-purge/task.yaml b/tests/main/postrm-purge/task.yaml index 4cbd01cbe2f..a9cfad2004c 100644 --- a/tests/main/postrm-purge/task.yaml +++ b/tests/main/postrm-purge/task.yaml @@ -1,10 +1,16 @@ summary: Check that package remove and purge removes everything related to snaps + details: | Ensure that the purge we do when removing the snapd package really removes all snap related files from the host system. + systems: [-ubuntu-core-*] prepare: | + if tests.info is-snapd-from-archive; then + tests.exec skip-test "This test is skipped when the snapd pkg isn't built from local" && exit 0 + fi + # TODO: unify this with tests/main/snap-mgmt/task.yaml # note: no need to unset these since this spread test purges snapd totally @@ -57,6 +63,8 @@ prepare: | fi restore: | + tests.exec is-skipped && exit 0 + #shellcheck source=tests/lib/pkgdb.sh . "$TESTSLIB/pkgdb.sh" if [ -e pkg-removed ]; then @@ -65,9 +73,13 @@ restore: | fi debug: | + tests.exec is-skipped && exit 0 + systemctl --no-legend --full | grep -E 'snap\..*\.(service|timer|socket|slice)' || true execute: | + tests.exec is-skipped && exit 0 + systemctl --no-legend --full | MATCH 'snap\..*\.(service|timer|socket|slice)' #shellcheck source=tests/lib/pkgdb.sh diff --git a/tests/main/preseed-lxd/task.yaml b/tests/main/preseed-lxd/task.yaml index edc17fe7135..845bffda378 100644 --- a/tests/main/preseed-lxd/task.yaml +++ b/tests/main/preseed-lxd/task.yaml @@ -51,7 +51,12 @@ prepare: | # for images that are already preseeded, we need to undo the preseeding there echo "Running preseed --reset for already preseeded cloud images" - SNAPD_DEBUG=1 /usr/lib/snapd/snap-preseed --reset "$IMAGE_MOUNTPOINT" + SNAP_PRESEED=/usr/lib/snapd/snap-preseed + if tests.info is-snapd-from-archive; then + MOUNT_DIR="$(os.paths snap-mount-dir)" + SNAP_PRESEED="$MOUNT_DIR/snapd/current$SNAP_PRESEED" + fi + SNAPD_DEBUG=1 "$SNAP_PRESEED" --reset "$IMAGE_MOUNTPOINT" restore: | if [ -f remove-lxd ]; then @@ -101,16 +106,45 @@ execute: | lxc config device add my-ubuntu mounted-ubuntu-image disk source="$IMAGE_MOUNTPOINT" path="$IMAGE_MOUNTPOINT" lxc restart my-ubuntu - echo "Make latest snap-preseed command available in the lxd container" + echo "Prepare container to run snap-preseed command" lxc exec my-ubuntu -- mkdir -p "$GOHOME" - lxc file push /usr/lib/snapd/snap-preseed "my-ubuntu/$GOHOME/" lxc file push cloudimg.img "my-ubuntu/$GOHOME/" lxc file push preseed-prepare.sh "my-ubuntu/$GOHOME/" - - echo "Running preseeding in the lxd container" # mount /dev, /proc, /sys under mounted ubuntu image in the container. lxc exec my-ubuntu -- "$GOHOME"/preseed-prepare.sh "$IMAGE_MOUNTPOINT" - lxc exec my-ubuntu -- "$GOHOME"/snap-preseed "$IMAGE_MOUNTPOINT" + + echo "Make latest snap-preseed command available in the lxd container" + echo "Running preseeding in the lxd container" + SNAP_PRESEED=/usr/lib/snapd/snap-preseed + if tests.info is-snapd-from-archive; then + # We use the snap preseed from the current snapd snap + # The current snapd snap is installed in the lxd container to make + # sure all the dependencies are in place, otherwise it fails to access to + # /snap/snapd/current/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 + + CURRENT_SNAPD_SNAP="$(ls "$SNAPD_WORK_DIR"/snapd_snap/snapd_*.snap)" + CURRENT_SNAPD_SNAP_NAME="$(basename "$CURRENT_SNAPD_SNAP")" + if [ "$(wc -w <<< "$CURRENT_SNAPD_SNAP")" != 1 ]; then + echo "It is expected just 1 snapd snap in $SNAPD_WORK_DIR/snapd_snap" + exit 1 + fi + lxc file push "$CURRENT_SNAPD_SNAP" "my-ubuntu/$GOHOME/" + for iter in $(seq 30); do + echo "checking snapd ready in container: iter $iter" + if lxc exec my-ubuntu -- sh -c 'snap changes' | MATCH "Done.*Initialize system state"; then + break + fi + sleep 1 + done + lxc exec my-ubuntu -- snap install --dangerous "$GOHOME/$CURRENT_SNAPD_SNAP_NAME" + + MOUNT_DIR="$(os.paths snap-mount-dir)" + SNAP_PRESEED="$MOUNT_DIR/snapd/current$SNAP_PRESEED" + lxc exec my-ubuntu -- "$SNAP_PRESEED" "$IMAGE_MOUNTPOINT" + else + lxc file push "$SNAP_PRESEED" "my-ubuntu/$GOHOME/" + lxc exec my-ubuntu -- "$GOHOME"/snap-preseed "$IMAGE_MOUNTPOINT" + fi echo "Precondition check of mount units" # preseeded image is mounted on the host, so we can directly inspect diff --git a/tests/main/preseed/task.yaml b/tests/main/preseed/task.yaml index dff40f499c7..dcfdbb9e423 100644 --- a/tests/main/preseed/task.yaml +++ b/tests/main/preseed/task.yaml @@ -35,7 +35,11 @@ prepare: | # for images that are already preseeded, we need to undo the preseeding there echo "Running preseed --reset for already preseeded cloud images" - SNAPD_DEBUG=1 /usr/lib/snapd/snap-preseed --reset "$IMAGE_MOUNTPOINT" + SNAP_PRESEED=/usr/lib/snapd/snap-preseed + if tests.info is-snapd-from-archive; then + SNAP_PRESEED="/snap/snapd/current$SNAP_PRESEED" + fi + SNAPD_DEBUG=1 "$SNAP_PRESEED" --reset "$IMAGE_MOUNTPOINT" restore: | #shellcheck source=tests/lib/preseed.sh @@ -48,11 +52,16 @@ execute: | LXD_BASE_SNAP=core22 fi + SNAP_PRESEED=/usr/lib/snapd/snap-preseed + if tests.info is-snapd-from-archive; then + SNAP_PRESEED="/snap/snapd/current$SNAP_PRESEED" + fi + echo "Checking missing chroot path arg error" - /usr/lib/snapd/snap-preseed 2>&1 | MATCH "error: need chroot path as argument" + "$SNAP_PRESEED" 2>&1 | MATCH "error: need chroot path as argument" echo "Running pre-seeding" - SNAPD_DEBUG=1 /usr/lib/snapd/snap-preseed "$IMAGE_MOUNTPOINT" + SNAPD_DEBUG=1 "$SNAP_PRESEED" "$IMAGE_MOUNTPOINT" # precondition, core snap mounted by snap-preseed got unmounted mount | NOMATCH "snap-preseed" diff --git a/tests/main/searching/task.yaml b/tests/main/searching/task.yaml index 1806af82477..2da2fd000a9 100644 --- a/tests/main/searching/task.yaml +++ b/tests/main/searching/task.yaml @@ -15,6 +15,8 @@ backends: [-autopkgtest] systems: [-ubuntu-*-ppc64el, -ubuntu-*-s390x] execute: | + tests.exec is-skipped && exit 0 + echo "List all featured snaps" expected='(?s).*Name +Version +Publisher +Notes +Summary *\n(.*?\n)?.*' snap find > featured.txt diff --git a/tests/main/security-device-cgroups-helper/task.yaml b/tests/main/security-device-cgroups-helper/task.yaml index ad6321a5a7f..9a9ecb5ad16 100644 --- a/tests/main/security-device-cgroups-helper/task.yaml +++ b/tests/main/security-device-cgroups-helper/task.yaml @@ -24,7 +24,14 @@ debug: | execute: | #shellcheck source=tests/lib/systems.sh . "$TESTSLIB/systems.sh" + + # In essence we need to use the helper from snap, as it's the only location + # which is now carrying the potentially modified code (since snapd package + # is installed from distribution repos and so it contains older, already merged code). libexecdir=$(os.paths libexec-dir) + if tests.info is-snapd-from-archive; then + libexecdir="/snap/snapd/current$libexecdir" + fi echo "Given a snap is installed" "$TESTSTOOLS"/snaps-state install-local test-strict-cgroup-helper diff --git a/tests/main/snap-confine-tmp-mount/task.yaml b/tests/main/snap-confine-tmp-mount/task.yaml index 6753730a203..79948a47e84 100644 --- a/tests/main/snap-confine-tmp-mount/task.yaml +++ b/tests/main/snap-confine-tmp-mount/task.yaml @@ -24,7 +24,11 @@ debug: | cat /tmp/snap-confine-stderr.log || true execute: | + MOUNT_DIR="$(os.paths snap-mount-dir)" SNAP_CONFINE=$(os.paths libexec-dir)/snapd/snap-confine + if tests.info is-snapd-from-archive; then + SNAP_CONFINE="$MOUNT_DIR/snapd/current$SNAP_CONFINE" + fi # on Ubuntu Core we need to use the correct path to ensure it is # appropriately confined by apparmor as it may be from the snapd diff --git a/tests/main/snap-run-devmode-classic/task.yaml b/tests/main/snap-run-devmode-classic/task.yaml index 47cc5fdedf9..d866f70b62f 100644 --- a/tests/main/snap-run-devmode-classic/task.yaml +++ b/tests/main/snap-run-devmode-classic/task.yaml @@ -19,10 +19,6 @@ systems: # when the host has cgroupsv2 in it - ubuntu-16.04-* -# Start early as it takes a long time. -priority: 100 -kill-timeout: 60m - environment: # ensure that re-exec is on by default like it should be SNAP_REEXEC: "1" @@ -39,28 +35,13 @@ environment: BASE_NON_CORE_STRICT_SNAP: test-snapd-sh-core18 prepare: | - # much of what follows is copied from tests/main/snapd-snap - - apt-get autoremove -y lxd lxd-client - - # load the fuse kernel module before installing lxd - modprobe fuse - "$TESTSTOOLS"/lxd-state prepare-snap - - # install snapcraft snap - - snap install snapcraft --channel="${SNAPCRAFT_SNAP_CHANNEL}" --classic - tests.cleanup defer snap remove --purge snapcraft - - # shellcheck disable=SC2164 - pushd "${PROJECT_PATH}" - echo "Build the snap" - snap run snapcraft --verbosity debug --output snapd-from-branch.snap - # Save some disk - snap run snapcraft clean - popd + # Core snap is not modified for classic re-exec any more in this scenario + if tests.info is-snapd-from-archive; then + tests.exec skip-test "Core snap is not modified for classic re-exec any more in this scenario" && exit 0 + fi - mv "$PROJECT_PATH/snapd-from-branch.snap" "$PWD/snapd-from-branch.snap" + CURRENT_SNAPD_SNAP="$(ls "$SNAPD_WORK_DIR"/snapd_snap/snapd_*.snap)" + cp "$CURRENT_SNAPD_SNAP" "$PWD/snapd-from-branch.snap" # now repack the core snap with this snapd snap snap download core --edge --basename=core-from-edge @@ -68,7 +49,7 @@ prepare: | coredir=edge-core-snap snapddir=snapd-from-branch - dpkg-deb -x "${SPREAD_PATH}"/../snapd_*.deb "${snapddir}" + dpkg-deb -x "$GOHOME"/snapd_*.deb "${snapddir}" pushd "${PROJECT_PATH}" SNAPD_SNAP_VERSION=$(./mkversion.sh --output-only) @@ -87,11 +68,11 @@ prepare: | rm -r "$coredir" rm -r "$snapddir" -restore: | - "$TESTSTOOLS"/lxd-state undo-mount-changes || true - execute: | + tests.exec is-skipped && exit 0 + if [ "$SNAP_TO_USE_FIRST" = "core" ]; then + # first install our core snap because we don't have the snapd snap on # the system yet, so we don't need to do any shenanigans snap install --dangerous core-from-branch.snap @@ -141,7 +122,6 @@ execute: | # we already had the core snap installed, so we need to purge things # and then install only the snapd snap to test this scenario - snap remove go snapcraft snap remove core18 apt remove --purge -y snapd apt install snapd -y @@ -186,8 +166,5 @@ execute: | fi # undo the purging - apt install -y "$PROJECT_PATH/../"snapd_1337.*_amd64.deb - else - echo "unknown variant $SNAP_TO_USE_FIRST" - exit 1 + apt install -y "$GOHOME"/snapd_*.deb fi diff --git a/tests/main/snap-run-hook/task.yaml b/tests/main/snap-run-hook/task.yaml index d73c13e7ab2..166de84a86f 100644 --- a/tests/main/snap-run-hook/task.yaml +++ b/tests/main/snap-run-hook/task.yaml @@ -12,6 +12,9 @@ environment: ENVDUMP: /var/snap/basic-hooks/current/hooks-env prepare: | + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi "$TESTSTOOLS"/snaps-state install-local basic-hooks SNAP_MOUNT_DIR="$(os.paths snap-mount-dir)" if [ "$SNAP_REEXEC" = "1" ] && [ "$SNAP_MOUNT_DIR" != "/snap" ] && [ ! -L /snap ]; then @@ -22,6 +25,8 @@ prepare: | fi execute: | + tests.exec is-skipped && exit 0 + # Note that `snap run` doesn't exit non-zero if the hook is missing, so we # check the output instead. diff --git a/tests/main/snap-run/task.yaml b/tests/main/snap-run/task.yaml index 3f3ff0e4ef1..8c2a25c2880 100644 --- a/tests/main/snap-run/task.yaml +++ b/tests/main/snap-run/task.yaml @@ -14,13 +14,19 @@ environment: STRACE_STATIC_CHANNEL: candidate prepare: | + tests.exec is-skipped && exit 0 + "$TESTSTOOLS"/snaps-state install-local basic-run "$TESTSTOOLS"/snaps-state install-local test-snapd-sh debug: | + tests.exec is-skipped && exit 0 + cat stderr || true execute: | + tests.exec is-skipped && exit 0 + echo "Running a trivial command causes no DENIED messages" test-snapd-sh.sh -c 'echo hello' dmesg | not grep DENIED diff --git a/tests/main/snap-seccomp/task.yaml b/tests/main/snap-seccomp/task.yaml index e0e07e8d1b2..03ce9a600dd 100644 --- a/tests/main/snap-seccomp/task.yaml +++ b/tests/main/snap-seccomp/task.yaml @@ -22,6 +22,11 @@ execute: | snap install test-snapd-sh test-snapd-sh.sh -c 'echo hello' | MATCH hello + if tests.info is-snapd-from-archive; then + MOUNT_DIR="$(os.paths snap-mount-dir)" + SNAP_SECCOMP="$MOUNT_DIR/snapd/current$SNAP_SECCOMP" + fi + # from the old test_complain echo "Test that the @complain keyword works" rm -f "${PROFILE}.bin2" diff --git a/tests/main/snapd-homedirs-vendored/task.yaml b/tests/main/snapd-homedirs-vendored/task.yaml index 918aca507c8..3ee097ce141 100644 --- a/tests/main/snapd-homedirs-vendored/task.yaml +++ b/tests/main/snapd-homedirs-vendored/task.yaml @@ -7,12 +7,16 @@ details: | # On ubuntu 18 and below snapd-internal is not available # ubuntu-24*: the snapd deb is version 2.60.4 which is post-vendored apparmor. -systems: [ubuntu-18*, ubuntu-20*, ubuntu-22*, ubuntu-23*] +systems: [ubuntu-18*, ubuntu-20*, ubuntu-22*] environment: USERNAME: home-sweet-home prepare: | + if tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi + # Create a new user in a non-standard location mkdir -p /remote/users useradd -b /remote/users -m -U "$USERNAME" @@ -28,6 +32,8 @@ prepare: | rm -rf "${unpackdir}" restore: | + tests.exec is-skipped && exit 0 + userdel -f --remove "$USERNAME" rm -rf /remote/users @@ -38,6 +44,8 @@ restore: | distro_install_build_snapd debug: | + tests.exec is-skipped && exit 0 + # output custom snap-confine snippets ls -l /var/lib/snapd/apparmor/snap-confine/ for f in /var/lib/snapd/apparmor/snap-confine/*; do @@ -46,6 +54,8 @@ debug: | done execute: | + tests.exec is-skipped && exit 0 + echo "Downgrading the snapd deb to pre-vendored apparmor times" TARGET_VER="$(apt list -a snapd | grep -- -updates | cut -f2 -d' ')" # Ubuntu release on development doesn't have updates @@ -58,7 +68,6 @@ execute: | echo "But installing the vendored apparmor snapd with our changes" snap install --dangerous snapd_modified.snap - # Verify supported features snap debug sandbox-features --required apparmor:parser:snapd-internal snap debug sandbox-features --required apparmor:parser:include-if-exists diff --git a/tests/main/snapd-reexec/task.yaml b/tests/main/snapd-reexec/task.yaml index c09a1627a2a..76a9bb2d596 100644 --- a/tests/main/snapd-reexec/task.yaml +++ b/tests/main/snapd-reexec/task.yaml @@ -77,7 +77,19 @@ execute: | fi echo "Ensure we re-exec by default" - /usr/bin/env SNAPD_DEBUG=1 snap list 2>&1 | MATCH "DEBUG: restarting into \"$SNAPD_MOUNT_DIR/current/usr/bin/snap\"" + IS_REEXEC=true + if [ "$SNAPD_SRC" = core ] && tests.info is-snapd-from-archive; then + # In this scenario it is possible that the core snap version is older + # than the distribution package because the core snap comes from latest/edge + # which is not updated regularly. + # TODO: Compare manually the snap and deb versions and determine which string needs to be in MATCH + if /usr/bin/env SNAPD_DEBUG=1 snap list 2>&1 | MATCH "snap \(at .*\) is older \(.*\) than distribution package"; then + IS_REEXEC=false + fi + fi + if [ "$IS_REEXEC" = true ]; then + /usr/bin/env SNAPD_DEBUG=1 snap list 2>&1 | MATCH "DEBUG: restarting into \"$SNAPD_MOUNT_DIR/current/usr/bin/snap\"" + fi echo "Ensure that we do not re-exec into older versions" systemctl stop snapd.service snapd.socket @@ -118,11 +130,13 @@ execute: | systemctl daemon-reload systemctl start snapd.service snapd.socket - echo "Ensure SNAP_REEXEC=0 is honored for snap" - mount --bind /tmp/broken-snapd "$SNAPD_MOUNT_DIR/current/usr/bin/snap" - snap list|MATCH "from the $SNAPD_SRC snap" + if [ "$IS_REEXEC" = true ]; then + echo "Ensure SNAP_REEXEC=0 is honored for snap" + mount --bind /tmp/broken-snapd "$SNAPD_MOUNT_DIR/current/usr/bin/snap" + snap list | MATCH "from the $SNAPD_SRC snap" + umount "$SNAPD_MOUNT_DIR/current/usr/bin/snap" + fi SNAP_REEXEC=0 SNAPD_DEBUG=1 snap list 2>&1 |MATCH "DEBUG: re-exec disabled by user" - umount "$SNAPD_MOUNT_DIR/current/usr/bin/snap" echo "Ensure a snapd source refresh restarts snapd" prev_src=$(snap list | awk "/^$SNAPD_SRC / {print(\$3)}") @@ -135,8 +149,12 @@ execute: | echo "Test broken $now_src and $prev_src are the same" exit 1 fi - SNAPD_PATH=$(readlink -f "/proc/$(pidof snapd)/exe") - if [ "$SNAPD_PATH" != "/snap/$SNAPD_SRC/${now_src}/usr/lib/snapd/snapd" ]; then - echo "unexpected $SNAPD_PATH for $now_src snap (previous $prev_src)" - exit 1 - fi + + # This check is valid only if snap command is re-executing to the snap + if [ "$IS_REEXEC" = true ]; then + SNAPD_PATH=$(readlink -f "/proc/$(pidof snapd)/exe") + if [ "$SNAPD_PATH" != "/snap/$SNAPD_SRC/${now_src}/usr/lib/snapd/snapd" ]; then + echo "unexpected $SNAPD_PATH for $now_src snap (previous $prev_src)" + exit 1 + fi + fi diff --git a/tests/main/snapd-snap-transition-auto-install/task.yaml b/tests/main/snapd-snap-transition-auto-install/task.yaml index 26fbb6bb49e..5c14a5ece7a 100644 --- a/tests/main/snapd-snap-transition-auto-install/task.yaml +++ b/tests/main/snapd-snap-transition-auto-install/task.yaml @@ -14,7 +14,11 @@ environment: SNAPD_SRC/deb: "deb" SNAPD_SRC/core: "core" -prepare: | +prepare: | + if tests.info is-snapd-from-archive; then + tests.exec skip-test "This test needs snapd deb built from local" && exit 0 + fi + # Expect system preparation stage installed built snapd and core snaps. # Check this and remove all snaps to remove snapd. snap list core @@ -45,6 +49,8 @@ prepare: | not snap list snapd execute: | + tests.exec is-skipped && exit 0 + # check re-exec disabled, exec from snapd deb or re-exec from core snap snap_mount_dir="$(os.paths snap-mount-dir)" expect="" diff --git a/tests/main/snapd-snap/task.yaml b/tests/main/snapd-snap/task.yaml index af285889cd4..2ab84ea8b15 100644 --- a/tests/main/snapd-snap/task.yaml +++ b/tests/main/snapd-snap/task.yaml @@ -114,7 +114,7 @@ prepare: | # When the SRU validation tests are executed, the core snap is not repacked # and the var MODIFY_CORE_SNAP_FOR_REEXEC is 0 - if [ "$MODIFY_CORE_SNAP_FOR_REEXEC" = 1 ]; then + if [ "$MODIFY_CORE_SNAP_FOR_REEXEC" = 1 ] && ! tests.info is-snapd-from-archive; then CORE_SNAP_REV=$(snap list core | grep -v Name | awk '{print $3}') CORE_SNAP="/var/lib/snapd/snaps/core_${CORE_SNAP_REV}.snap" ORIG_CORE_SNAP=${CORE_SNAP}.orig @@ -372,7 +372,8 @@ execute: | fi # reboot to ensure snapd.apparmor still works then too - if [ "$SPREAD_REBOOT" = "0" ]; then + + if [ "$SPREAD_REBOOT" = "0" ] && ! tests.info is-snapd-from-archive; then # downgrade the snapd deb from the distro package to test that we can # still handle the generated apparmor profiles etc from the snapd snap echo "Downgrading snapd to distro packaged version..." @@ -381,7 +382,7 @@ execute: | # not have credentials for esm.ubuntu.com if ! os.query is-xenial; then apt install -y --allow-downgrades "snapd/$(lsb_release -sc)" - tests.cleanup defer apt install -y "$PROJECT_PATH/../"snapd_1337.*_"$(dpkg-architecture -qDEB_HOST_ARCH)".deb + tests.cleanup defer apt install -y "$GOHOME"/snapd_*.deb # check snapd.apparmor is still working after downgrade if ! os.query is-xenial; then diff --git a/tests/main/snapd-update-services/task.yaml b/tests/main/snapd-update-services/task.yaml index a66fe67181d..b6ab7c360d5 100644 --- a/tests/main/snapd-update-services/task.yaml +++ b/tests/main/snapd-update-services/task.yaml @@ -21,17 +21,12 @@ prepare: | # Download current snapd edge snap download --edge snapd --basename=snapd_edge - # Repack it with currently built snapd - unpackdir=/tmp/snapd-update-services - unsquashfs -no-progress -d "${unpackdir}" snapd_edge.snap - dpkg-deb -x "${GOHOME}"/snapd_*.deb "${unpackdir}" - snap pack "${unpackdir}" --filename snapd_modified.snap - rm -rf "${unpackdir}" + # Get the snapd package built from the current branch + CURRENT_SNAPD_SNAP="$(ls "$SNAPD_WORK_DIR"/snapd_snap/snapd_*.snap)" + cp "$CURRENT_SNAPD_SNAP" "$PWD/snapd_current.snap" restore: | rm -rf /tmp/snapd-update-services - rm -f snapd_edge.snap - rm -f snapd_modified.snap execute: | snap install --dangerous snapd_edge.snap @@ -69,7 +64,7 @@ execute: | done # Snapd update - snap install --dangerous snapd_modified.snap + snap install --dangerous snapd_current.snap # Check the service file has been updated systemctl is-active snap.test-snapd-service.test-snapd-service.service diff --git a/tests/main/store-state/task.yaml b/tests/main/store-state/task.yaml index 554dd247d20..dd61ccf3bfa 100644 --- a/tests/main/store-state/task.yaml +++ b/tests/main/store-state/task.yaml @@ -28,13 +28,17 @@ execute: | "$TESTSTOOLS"/store-state -h | MATCH "usage: store-state setup-fake-store " "$TESTSTOOLS"/store-state --help | MATCH "usage: store-state setup-fake-store " - # Setup staging store - "$TESTSTOOLS"/store-state setup-staging-store - snap info core | MATCH "store-url:.*https://staging-api.snapcraft.io" - - # Teardown staging store - "$TESTSTOOLS"/store-state teardown-staging-store - snap info core | MATCH "store-url:.*https://snapcraft.io" + # Staging store cannot be used with snapd deb from the repository + # The staging keys are being checked before snapd re-exec into snapd snap + if not tests.info is-snapd-from-archive; then + # Setup staging store + "$TESTSTOOLS"/store-state setup-staging-store + snap info core | MATCH "store-url:.*https://staging-api.snapcraft.io" + + # Teardown staging store + "$TESTSTOOLS"/store-state teardown-staging-store + snap info core | MATCH "store-url:.*https://snapcraft.io" + fi # Setup fakestore STORE_DIR="$(pwd)/fake-store-blobdir" diff --git a/tests/main/try/task.yaml b/tests/main/try/task.yaml index 5a982020a94..725714d0dee 100644 --- a/tests/main/try/task.yaml +++ b/tests/main/try/task.yaml @@ -15,15 +15,21 @@ environment: SERVICE_NAME: "test-service" prepare: | + tests.exec is-skipped && exit 0 + # shellcheck source=tests/lib/network.sh . "$TESTSLIB"/network.sh make_network_service "$SERVICE_NAME" "$PORT" restore: | + tests.exec is-skipped && exit 0 + systemctl stop "$SERVICE_NAME" rm -f "$READABLE_FILE" execute: | + tests.exec is-skipped && exit 0 + echo "Given a buildable snap in a known directory" echo "When try is executed on that directory" snap try "$TESTSLIB"/snaps/test-snapd-tools diff --git a/tests/main/upgrade-from-release/task.yaml b/tests/main/upgrade-from-release/task.yaml index 76323588ea5..dcaf61ee2bd 100644 --- a/tests/main/upgrade-from-release/task.yaml +++ b/tests/main/upgrade-from-release/task.yaml @@ -67,7 +67,7 @@ execute: | if [ "${VERSION_ID}" = "14.04" ]; then dpkg -i "$GOHOME"/snapd*.deb else - apt install -y "$GOHOME"/snapd_1337*.deb + apt install -y "$GOHOME"/snapd_*.deb fi echo "snapd listens to requests" diff --git a/tests/main/user-session-env/task.yaml b/tests/main/user-session-env/task.yaml index 550c2f51631..c7b81bb0010 100644 --- a/tests/main/user-session-env/task.yaml +++ b/tests/main/user-session-env/task.yaml @@ -12,7 +12,7 @@ systems: - -amazon-linux-2-* # no fish package for AMZN2 - -amazon-linux-2023-* # no fish package for AL2023 - -centos-9-* # no fish package - + - -debian-11-* # not supposed to work as snapd is very old in the repository environment: TEST_ZSH_USER: test-zsh @@ -54,13 +54,18 @@ execute: | su -c 'sh -c "exec env"' -l "$TEST_FISH_USER" > "${TEST_FISH_USER}-child-env" SNAP_MOUNT_DIR="$(os.paths snap-mount-dir)" + LOCAL_PATH= + if tests.info is-snapd-from-archive; then + MOUNT_DIR="$(os.paths snap-mount-dir)" + LOCAL_PATH="$MOUNT_DIR/snapd/current" + fi for user in test "$TEST_ZSH_USER" "$TEST_FISH_USER" ; do echo "checking $user" if [ -e "${user}-session-env" ]; then # Even though there's user session support, systemd may be too old and # not support user-environment-generators (specifically systemd versions # earlier than 233). - if [ -d /usr/lib/systemd/user-environment-generators ]; then + if [ -d "$LOCAL_PATH"/usr/lib/systemd/user-environment-generators ]; then MATCH 'XDG_DATA_DIRS=.*[:]?/var/lib/snapd/desktop[:]?.*' < "${user}-session-env" MATCH "PATH=.*[:]?${SNAP_MOUNT_DIR}/bin[:]?.*" < "${user}-session-env" fi diff --git a/tests/main/writable-areas/task.yaml b/tests/main/writable-areas/task.yaml index 4c0638bd90b..7ead233e1d9 100644 --- a/tests/main/writable-areas/task.yaml +++ b/tests/main/writable-areas/task.yaml @@ -13,6 +13,10 @@ environment: SNAP_REEXEC/reexec1: 1 prepare: | + # No needed to test the snap pkg when it is coming from the repository + if [ "$SNAP_REEXEC" = "0" ] && tests.info is-snapd-from-archive; then + tests.exec skip-test "No needed to test the snap pkg when it is coming from the repository" && exit 0 + fi snap pack data-writer SNAP_MOUNT_DIR="$(os.paths snap-mount-dir)" if [ "$SNAP_REEXEC" = "1" ] && [ "$SNAP_MOUNT_DIR" != "/snap" ] && [ ! -L /snap ]; then @@ -23,6 +27,8 @@ prepare: | fi execute: | + tests.exec is-skipped && exit 0 + snap install --dangerous data-writer_1.0_all.snap echo "Apps can write to writable areas" diff --git a/tests/regression/lp-1871652/task.yaml b/tests/regression/lp-1871652/task.yaml index 7c0f0d8c96e..f2238d874c9 100644 --- a/tests/regression/lp-1871652/task.yaml +++ b/tests/regression/lp-1871652/task.yaml @@ -43,9 +43,10 @@ prepare: | # system. Given that snapd on the system has just been built from source it # will have the custom 1337 version string. # NOTE: For SRU validation snapd is not built from source, so version does not match 1337 + # NOTE: When snapd is downloaded from the repo version does not match 1337 # NOTE: This step is only necessary while snapd in the store is older than the fix lxc file push /usr/bin/snap bionic/usr/bin/snap - if [ ! "$SRU_VALIDATION" = 1 ]; then + if [ ! "$SRU_VALIDATION" = 1 ] && ! tests.info is-snapd-from-archive ; then lxc exec bionic -- snap version | MATCH 1337 fi diff --git a/tests/smoke/find-info/task.yaml b/tests/smoke/find-info/task.yaml index dafa1d04493..98181a0f3db 100644 --- a/tests/smoke/find-info/task.yaml +++ b/tests/smoke/find-info/task.yaml @@ -6,9 +6,10 @@ details: | display detailed information about it by using the "snap info" command. execute: | + tests.exec is-skipped && exit 0 + echo "Ensure 'snap find' works" snap find test-snapd-tools | MATCH ^test-snapd-tools echo "Ensure we can see useful info for it" snap info test-snapd-tools | MATCH '^name:\ +test-snapd-tools' - diff --git a/tests/smoke/install/task.yaml b/tests/smoke/install/task.yaml index 3e1549b0953..7603d3d22f3 100644 --- a/tests/smoke/install/task.yaml +++ b/tests/smoke/install/task.yaml @@ -10,18 +10,24 @@ details: | snaps: core, core18, core20 and core22. restore: | + tests.exec is-skipped && exit 0 + rm -f /home/test/stderr.log rm -f /home/test/stdout.log # required! in autopkgtest no suite restore is run at all snap remove --purge test-snapd-sh debug: | + tests.exec is-skipped && exit 0 + if test -e stderr.log; then echo "content of stderr.log" cat stderr.log fi execute: | + tests.exec is-skipped && exit 0 + #shellcheck source=tests/lib/systems.sh . "$TESTSLIB"/systems.sh diff --git a/tests/smoke/remove/task.yaml b/tests/smoke/remove/task.yaml index a7fa6299b6c..946c9124c5b 100644 --- a/tests/smoke/remove/task.yaml +++ b/tests/smoke/remove/task.yaml @@ -7,6 +7,8 @@ details: | removal no leftovers remain in the mounting area. execute: | + tests.exec is-skipped && exit 0 + "$TESTSTOOLS"/snaps-state install-local test-snapd-sh SNAP_MOUNT_DIR="$(os.paths snap-mount-dir)" diff --git a/tests/smoke/sandbox/task.yaml b/tests/smoke/sandbox/task.yaml index f11164e80db..442cc3bb97c 100644 --- a/tests/smoke/sandbox/task.yaml +++ b/tests/smoke/sandbox/task.yaml @@ -9,12 +9,16 @@ details: | the CPU supports one. restore: | + tests.exec is-skipped && exit 0 + rm -f /home/test/foo # required! in autopkgtest no suite restore is run at all snap remove --purge test-snapd-sandbox snap remove --purge test-snapd-hello-multi-arch execute: | + tests.exec is-skipped && exit 0 + if [ "$(snap debug confinement)" != "strict" ]; then if os.query is-ubuntu; then echo "all ubuntu systems must have strict confinement" diff --git a/tests/smoke/versioning/task.yaml b/tests/smoke/versioning/task.yaml index f4b5a870354..71d44649dd6 100644 --- a/tests/smoke/versioning/task.yaml +++ b/tests/smoke/versioning/task.yaml @@ -9,6 +9,8 @@ details: | modified the code in unintended ways. execute: | + tests.exec is-skipped && exit 0 + # TODO: fix dirty version number on riscv echo "Ensure the version number is not 'dirty'" if not uname -m | MATCH riscv64; then