snap-bootstrap: experimental: handling install/factory reset steps within snap-bootstrap #12382

kubiko · 2022-12-01T02:02:38Z

This is a reference PR implementing install and factory-reset modes within snap-bootstrap
It should server also as tracking for the work done so far.
This activity is split across multiple PRs and not all might be mentioned here, especially if they have already landed.
As such, this PR can be regularly rebased and force-pushed to update tracking.

Current state: install mode support had landed on the master branch

Related PRs:
Open:
#12641

Landed:
#12846

Keeping original descriptions around for reference:
Original description:

Experimental attempt to handle install step within snap-bootstrap, effectively compressing initial install into a single boot.

Since the install step relies on additional tools to manage partitions, create file systems, set up encryption, this change has to be also accompanied by additional tools in initramfs.
As a reference of initramfs size increases:

additional tools and extra features in snap-bootstrap (sec tools) resulted in 2.3MB size increase for UC22 (arm64, zstd compression)

TODOs:
Improve seed meta loading. We load seed meta 3 times(assuming preseeded image)

initial essential snaps for install, hashes: kernel, base, gadget

handling preseed, hashes: all

post-install snapd handling, hashes: snapd
Since we support parallelism we should ideally:

load all the essentials (kernel, core, snapd, gadget) most systems are either single and 4+ cores

reuse the hash of essential snaps and only hash other snaps

pedronis

this looks promising, we need to look into:

when do we do this? always? can we? not always but based on what?
- does that mean that install handlers go away?
what to do about hooks, calling fde-reveal-key instead of fde-setup is a change of contract that doesn't make sense, also it means we cannot really do this always
we need to properly split the parts of devicestate we are reusing to a package (to avoid bringing in most of overlord in snap-bootstrap), and also try to have even more higher-level helpers, there is still quite a bit of stuff that feels repeated from install code and could go out of sync

we need to see what do about places that look at the kernel command line directly, I could spot:

https://github.com/snapcore/core-base/search?q=ConditionKernelCommandLine

./overlord/configstate/configcore/services.go:		mode, _, _ := boot.ModeAndRecoverySystemFromKernelCommandLine()
./cmd/snap/cmd_auto_import.go:	mode, _, err := boot.ModeAndRecoverySystemFromKernelCommandLine()
./cmd/snap-bootstrap/cmd_initramfs_mounts.go:	mode, recoverySystem, err := boot.ModeAndRecoverySystemFromKernelCommandLine()

what to do about install-device hook

pedronis · 2022-12-02T09:09:24Z

cmd/snap-bootstrap/cmd_initramfs_mounts.go

@@ -149,7 +151,7 @@ func generateInitramfsMounts() (err error) {
 	case "recover":
 		err = generateMountsModeRecover(mst)
 	case "install":
-		err = generateMountsModeInstall(mst)
+		err = installNewSystemGenerateMounts(mst)


is the idea to always do this? should we do this only if preseed is setup?

it seems we can't do this in general because of the different hook contract ?

I have a rather radical opinion here :)
I thin we should completely eliminate "install boot" for all the scenarios unless proven otherwise that this is needed.
initramfs is exactly what we want as an ephemeral system to run install in.

as we discussed we have to:

start smaller and decide how to trigger this (which probably relates to the question about the hook contract)

we still have use cases for the full install mode (dedicated installers for example)

pedronis · 2022-12-02T09:12:11Z

cmd/snap-bootstrap/cmd_initramfs_mounts.go

+	// 1.1 load seed
+	deviceSeed, err := loadDeviceSeed(systemLabel)
+	if err != nil {
+		return err
+	}
+	// 1.2 load seed assertions
+	assertDb, err := loadSeedAssertions(deviceSeed)
+	if err != nil {
+		return err
+	}
+
+	// 1.3 load sead essential meta
+	if err = loadSeedEssentialMeta(deviceSeed, systemLabel); err != nil {
+		return err
+	}


this probably can be one helper?

pedronis · 2022-12-02T09:15:07Z

cmd/snap-bootstrap/cmd_initramfs_mounts.go

+	}
+
+	// 2.2 ensure next boot is to run mode
+	if err := boot.EnsureNextBootToRunMode(systemLabel); err != nil {


there is a bit more than this happening in doRestartSystemToRunMode that we might want/need to take care of

Actually, this should be removed from here and system should be marked as bootable later once it actually boots.
If we mark system runable here and it fails later, the install would not be re-run. Perhaps more fitting place for this would be:
https://github.com/snapcore/snapd/blob/master//boot/boot.go#L355

pedronis · 2022-12-02T09:16:25Z

cmd/snap-bootstrap/cmd_initramfs_mounts.go


+func generateSnapMountsModeRun(mst *initramfsMountsState, isClassic, hasSeedPart bool, rootfsDir string) error {


nitpick: this has a bit too many bool args

Ideally I'd extend initramfsMountsState struct to do more, so we do not pass this many things.
But fair point

pedronis · 2022-12-02T09:17:25Z