Diff snap shot on large memory fails

[dwaddington]

Hi,
When I try to diff snapshot on larger VMs e.g. 16G, a failure happens (smaller, e.g 8G are OK)..

2021-12-09T10:49:34.344708795 [anonymous-instance:fc_api] The API server received a Patch request on "/vm" with body "{\n            \"state\": \"Paused\"\n             }".
2021-12-09T10:49:34.344905332 [anonymous-instance:main] 'pause vm' VMM action took 63 us.
2021-12-09T10:49:34.344944946 [anonymous-instance:fc_api] The request was executed successfully. Status code: 204 No Content.
2021-12-09T10:49:34.344969903 [anonymous-instance:fc_api] 'pause vm' API request took 278 us.
2021-12-09T10:49:34.349022714 [anonymous-instance:fc_api] The API server received a Put request on "/snapshot/create" with body "{\n            \"snapshot_type\": \"Diff\",\n            \"snapshot_path\": \"./vm_state.snapdiff\",\n            \"mem_file_path\": \"./memory.snapdiff\",\n            \"version\": \"1.0.0\"\n    }".
2021-12-09T10:49:34.367118087 [anonymous-instance:main] Shutting down VM after intercepting signal 31, code 1.
2021-12-09T10:49:34.367161007 [anonymous-instance:main] Shutting down VM after intercepting a bad syscall (9).

What is happening here?

[alindima]

Hello, this is a seccomp failure, caused by the Firecracker process issuing a syscall that is not allowed by the installed filter, in this case it is mmap. I've managed to reproduce this and indeed the fix is allowing mmap with libc::MAP_ANONYMOUS | libc::MAP_PRIVATE on the vmm thread, called to allocate some large buffer (412 KB as far as I can see via strace).

Are you building Firecracker from the main branch?

As an experimental workaround you may compile your own seccomp filter using seccompiler-bin. More details here: https://github.com/firecracker-microvm/firecracker/blob/main/docs/seccomp.md#custom-filters-advanced-users-only

We will shortly add a patch on the main branch. Thanks for reporting this.

[dwaddington]

Thanks. Yes, I am building from the main branch. Let me look into the seccomp filter.

[alindima]

PR on main that includes the fix, as well as a regression test: #2819