Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support the creation of an empty snapshot #944

Closed
davidcassany opened this issue Oct 18, 2024 · 5 comments · Fixed by #964 · May be fixed by rancher/elemental-toolkit#2246
Closed

Support the creation of an empty snapshot #944

davidcassany opened this issue Oct 18, 2024 · 5 comments · Fixed by #964 · May be fixed by rancher/elemental-toolkit#2246
Assignees
@aschnell
Labels
enhancement

Comments

@davidcassany
Copy link

davidcassany commented Oct 18, 2024 •
edited
Loading

In snapper from an image builder perspective we are missing the capability of creating a new empty snapshot. By empty I mean creating a snapshot that is not based on a previous one.

The reasoning is the following, in Elemental, we aim to install/update images as snapshots with following sequence:

  1. Start transaction (new snapshot playground is created)
  2. include data into the new snapshot as desired
  3. Close transaction (make the new snapshot the new default root)

Pretty similar to what tukit from transactional updates does. The problem we have is that we implement this procedure for the very first snapshot too (our first root is considered to be a snapshot too like in MicroOS). We are basically missing a way to call snapper to create a new empty subvolume as an snapshot, that way, snapper could be really our wrapper around btrfs utility.

Let me expose how we create the first root to be seen as a snapshot by snapper (roughly translated into shell steps):

# /mnt is a btrfs partition
# Enable quota management and create subvolumes
btrfs quota enable /mnt
btrfs subvolume create /mnt/@
btrfs subvolume create /mnt/@/.snapshots

mkdir -p /mnt/@/.snapshots/1

btrfs subvolume snapshot /mnt/@ /mnt/@/.snapshots/1/snapshot

volid=$(btrfs subvolume list --sort path /mnt | head -n 1 | cut -d" " -f2)
btrfs subvolume set-default ${volid} /mnt

# Create snapshot 1 info
date=$(date +'%Y-%m-%d %H:%M:%S')
cat << EOF > /mnt/@/.snapshots/1/info.xml
<?xml version="1.0"?>
<snapshot>
  <type>single</type>
  <num>1</num>
  <date>${date}</date>
  <description>first root filesystem</description>
</snapshot>
EOF

# Dump content into the first snapshot subvolume

# set first snapshot as readonly
btrfs property set /mnt/@/.snapshots/1/snapshot ro true

The so the problem we have is that we need to wrap btrfs to create subvolumes, create snapshot metadata, handle subvolumes IDs vs snapshot IDs, etc. It feels a bit like we are re-implementing core functionalities of snapper itself.

What we'd love to see in snapper is something similar to:

# Create required /mnt/@/.snapshots subvolumes structure
snapper --configdir custom-cfg --root /mnt --no-dbus init

# Create a new empty snapshot, use --empty or something similar instead of '--from ID'
snapshotID=$(snapper --configdir custom-cfg --root /mnt --no-dbus create --empty --read-write --print-number --description "First root install" --userdata "first snapshot")

# Dump content into the first snapshot subvolume

# Set just created snapshot as read only
snapper --configdir custom-cfg --root /mnt --no-dbus modify --read-only $snapshotID

So to my eyes is essentially bringing to snapshot the capability to create empty snapshots (no parent) and the capability to provide custom configuration beyond /etc. This is good for imaging and creating systems like Micro OS in which the very first / is considered a snapshot already. This would also be valuable for KIWI to simplify some of the logic, as there in order to support Micro OS builds there is also btrfs custom logic to mimic snapper.
@aschnell
Copy link
Member

Creation of an empty snapshot is very easy since it is partly already implemented. YaST
also needs that functionality but it is so far only used in the "installation-helper".

https://github.com/openSUSE/snapper/blob/master/client/installation-helper/installation-helper.cc#L106

Since you use-case uses --no-dbus anyway it is not even required to extend the DBus
interface.

@aschnell aschnell self-assigned this Oct 24, 2024
@aschnell aschnell mentioned this issue Oct 25, 2024
Merged
This was referenced Oct 28, 2024
Merged
Merged
@aschnell
Copy link
Member

aschnell commented Dec 4, 2024

The problem is a bit tricky: So far first the snapper config has to be created and then the snapshot 1. When creating snapshot 1 as empty the snapper config is not included in the snapshot. Even when creating snapshot 1 as a snapshot of the toplevel subvolume the snapper config is also present in the toplevel subvolume (for which we already have a bug report btw).

I have now implemented a two step workflow: The first step "filesystem" creates the subvolumes. The second step "config" creates snapper files. More precise:

The step "filesystem" does the following:

create btrfs subvolume //.snapshots
create directory //.snapshots/1
create btrfs subvolume //.snapshots/1/snapshot
set default btrfs subvolume to //.snapshots/1/snapshot
create directory //.snapshots/1/snapshot/.snapshots

The step "config" does the following:

create snapper sysconfig //etc/sysconfig/snapper
create snapper config //etc/snapper/configs/root
create snapper info file //.snapshots/1/info.xml

The installer has to mount the filesystem before the step "filesystem". Between the two steps the filesystem must be remounted (since the default subvolume was changes). Additionally the .snapshots subvolume must be mounted by the installer. The installer must also handle /etc/fstab or similar.

Example workflow:

mkfs.btrfs -f /dev/sdc1

mount /dev/sdc1 /test

/usr/lib/snapper/installation-helper --root-prefix /test --step filesystem

umount /test

mount /dev/sdc1 /test
mount /dev/sdc1 -o subvol=.snapshots /test/.snapshots

/usr/lib/snapper/installation-helper --root-prefix /test --step config --description initial --userdata a=1

Now snapper can be used normally, e.g.:

snapper --no-dbus --root /test ls

Unfortunately I cannot test it thoroughly since I do not have an installer that can use this new workflow. So feedback is needed.

Test packages are available at https://build.opensuse.org/package/show/home:aschnell:snapper/snapper.

@aschnell aschnell mentioned this issue Dec 12, 2024
Merged
@aschnell aschnell linked a pull request Dec 12, 2024 that will close this issue
- added new workflow using only two steps to installation-helper #964
Merged
@davidcassany
Copy link
Author

davidcassany commented Dec 16, 2024 •
edited
Loading

@aschnell thanks much for these changes. I think this is fine, so far I could manage to run an installer without having to create any other subvolume than /@. If I understand this correctly, the @ parent subvolume is a convention unrelated to snapper, hence I assume this was not included as part of the filesystem step as from snapper PoV this is not a requirement. I guess the enabled quota I also unrelated to snapper.

My tests were successful so the only thing I miss now on my end is a version threshold to discriminate snapper including these changes. Whenever there is a new version cut in Factory we should be capable to actually finalize the draft I created for testing.

@aschnell
Copy link
Member

Thanks for the feedback.

And yes, the @ subvolume is optional. Some SUSE products have it, others not. In YaST it can be controlled via a product specific configuration file. I have tested both setups, see https://github.com/openSUSE/snapper/blob/master/client/installation-helper/test1.sh and https://github.com/openSUSE/snapper/blob/master/client/installation-helper/test2.sh.

I will make a new snapper release soon, likely January.

This was referenced Dec 17, 2024
Draft
Draft
@aschnell
Copy link
Member

I am making a 0.12.1 release right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aschnell aschnell

Labels
enhancement
Projects
None yet
Milestone
No milestone
2 participants
@aschnell @davidcassany