panic in systemd-nspawn container

[powersj]

Describe the bug
Importing github.com/awnumar/memguard causes a panic when run in a systemd-nspawn container.

To Reproduce

go.mod:

module test

go 1.19

require github.com/awnumar/memguard v0.22.3

main.go:

package main

import _ "github.com/awnumar/memguard"

func main() {}

Run:

go mod tidy
go run .

Expected behaviour
The command should complete with no panic.

Screenshots

$ go run .
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x49af1f]

goroutine 1 [running]:
github.com/awnumar/memguard/core.Purge.func1(0xc0000aea60)
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:23 +0x3f
github.com/awnumar/memguard/core.Purge()
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:51 +0x25
github.com/awnumar/memguard/core.Panic({0x4a69c0, 0xc000014370})
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:85 +0x25
github.com/awnumar/memguard/core.NewBuffer(0x20)
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/buffer.go:73 +0x2d5
github.com/awnumar/memguard/core.NewCoffer()
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/coffer.go:30 +0x34
github.com/awnumar/memguard/core.init.0()
	/root/go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/enclave.go:15 +0x2e
exit status 2

System (please complete the following information):

• OS and Kernel Versions: arch
• Memguard Version: v0.22.3
• Go Version: go1.19.4 linux/amd64

Additional context

We recently started using this project in telegraf in our new secret store implementation. A user reported this issue in influxdata/telegraf#12403 and I was able to reproduce it outside of telegraf with the above steps.

[awnumar]

Hey 👋 thanks for reporting this

Are you able to reproduce this issue outside of the container? What's the container image that's being used?

[powersj]

Are you able to reproduce this issue outside of the container?

I am not.

What's the container image that's being used?

The reporter in the Telegraf issue reported using Ubuntu 20.04. I am not sure if that is the host, the container, or both. @thomasmyn?

I reproduced using an arch container running on arch.

[thomasmyn]

Both, Security Updates from September 13, 2022
Ubuntu 20.04.5 LTS
5.4.0-135-generic x86_64

[antxxxx]

This also happens when running in a freebsd jail, but not when running on freebsd outside a jail. I can reproduce it using the example above.

Freebsd 13.1-RELEASE-p3
go version go1.19.4 freebsd/amd64

[antxxxx]

Cloning the repo and running go test also fails in a freebsd jail but not outside it

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x516b3f]

goroutine 1 [running]:
github.com/awnumar/memguard/core.Purge.func1(0xc000047a60)
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:23 +0x3f
github.com/awnumar/memguard/core.Purge()
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:51 +0x25
github.com/awnumar/memguard/core.Panic({0x56d960, 0xc00004a4e0})
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/exit.go:85 +0x25
github.com/awnumar/memguard/core.NewBuffer(0x20)
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/buffer.go:73 +0x2d5
github.com/awnumar/memguard/core.NewCoffer()
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/coffer.go:30 +0x34
github.com/awnumar/memguard/core.init.0()
        /root/go/pkg/mod/github.com/awnumar/[email protected]/core/enclave.go:15 +0x2e
exit status 2
FAIL    github.com/awnumar/memguard     0.004s

[antxxxx]

I found a solution for freebsd jail - you need to add allow.mlock to the jail when starting it.

I imagine its something similar for docker containers on linux - possibly adding IPC_LOCK capability to the container when starting it - see https://medium.com/@thejasongerard/resource-limits-mlock-and-containers-oh-my-cca1e5d1f259

[eljef]

I use systemd-nspawn containers to build software packages. I ran into this exact issue while building telegraf. I was able to solve the issue by adding --capability=CAP_IPC_LOCK as an argument to systemd-nspawn.

[eljef]

Adding the IPC_LOCK capability fixes the issue in my testing. If I explicitly deny the capability, I can reproduce the panic outside of a container.

It might be useful to document the requirement of the capability on Linux systems. I'm not sure how this would apply to BSDs or other UNIX systems. I don't believe anything is required on the Windows end. I could be wrong though.

[thomasmyn]

@eljef
thanks for the solution, this solved our problem

[powersj]

Hi @awnumar,

Sounds like the folks above discovered the necessary change in a nspawn. In terms of errors, is checking for this capability, versus causing a panic, something you would consider changing?

Thanks!

[eljef]

I'm unsure of the preferred implementation, but it does appear that there is a native capabilities package for Golang that exposes the capabilities for a process and allows for checking if IPC_LOCK is available in the effective flags set on Linux.

https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/cap#GetProc
https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/cap#Set.GetFlag

GetProc().GetFlag(cap.Effective, cap.IPC_LOCK)

[firelizzard18]

I also get this panic when trying to debug my process. It runs fine with go run ./my/cmd but it panics when I run it with Delve (including dlv debug ./my/cmd.

[awnumar]

@firelizzard18 for Delve specifically it looks like the issue was resolved

@powersj looks like an interesting approach, but it's probably not portable between platforms.