Use strings.Cut and strings.CutPrefix where possible #4470

kolyshkin · 2024-10-23T06:58:54Z

For the most part, this is a switch from strings.Split[N] to strings.Cut,
and from strings.HavePrefix/strings.TrimPrefix to strings.CutPrefix.

Using strings.Cut (added in Go 1.18, see 1) results in faster and
cleaner code with less allocations (as we're not using a slice).

Using strings.CutPrefix (added in Go 1.20, see 2) results in
slightly cleaner easier-to-read code.

There are a few other cleanups and nits here and there.
Please see individual commits for details.

rata · 2024-11-05T13:47:16Z

@kolyshkin I guess this is due to performance? Did you run some numbers or we just tested before this is faster and we are just using strings.Cut in more places? IMHO, it would be nice to say the reason on the commits.

If something breaks due to this, it would be nice to know that the commit just changed to this due to performance, so a revert is probably safe until a new fixed version is cooked. But that is not that easy to know if we don't explain why we change it in the commit.

kolyshkin · 2024-11-06T20:53:52Z

@kolyshkin I guess this is due to performance? Did you run some numbers or we just tested before this is faster and we are just using strings.Cut in more places? IMHO, it would be nice to say the reason on the commits.

If something breaks due to this, it would be nice to know that the commit just changed to this due to performance, so a revert is probably safe until a new fixed version is cooked. But that is not that easy to know if we don't explain why we change it in the commit.

Thanks, this makes sense. Individual commit messages as well as this PR description updated; PTAL.

kolyshkin · 2024-12-05T00:05:43Z

@opencontainers/runc-maintainers PTAL. This is a minor cleanup, aiming for more readable and faster code with less allocations.

lifubang · 2024-12-20T04:31:03Z

exec.go

-			return nil, fmt.Errorf("invalid --cgroup argument: %s", c)
-		}
-		if len(cs) == 1 { // no controller: prefix
+		if ctr, path, ok := strings.Cut(c, ":"); ok {


I havn't checked that whether this will lead to some regressions or not, for example, if c == "a:b:c:d:e":
When using SplitN, cs[1] will be "b";
When using Cut, path will be "b:c:d:e".

So, the old code would error out when c == "a:b:c:d:e".

The new code would not, assigning b:c:d:e to path.

To me, the new code is (ever so slightly) more correct, since : is a valid symbol which should be allowed in path.

Yup, was looking at that as well; agreed, looks like new code is more correct here.

I still need to have a quick look at the splitting on commas, and the if len(args) != 1 { check below. Probably will check out this branch locally

libcontainer/cgroups/fs/cpuacct.go

libcontainer/cgroups/fs2/defaultpath.go

thaJeztah · 2024-12-23T14:01:25Z

libcontainer/cgroups/fs2/psi.go

 		case "avg10":
 			pv = &data.Avg10
 		case "avg60":
 			pv = &data.Avg60
 		case "avg300":
 			pv = &data.Avg300
 		case "total":
-			v, err := strconv.ParseUint(kv[1], 10, 64)
+			v, err := strconv.ParseUint(val, 10, 64)


Not for this PR, but I started to look in some other codebases to dismantle these errors. The default error returned is something like;

invalid total PSI value: strconv.ParseUint: parsing "some invalid value": invalid syntax

Which tends to be a bit "implementation detail", and strconv.ParseUint is not very relevant to the user. So in some cases, dismantling the error can be useful; https://cs.opensource.google/go/go/+/refs/tags/go1.23.4:src/strconv/atoi.go;l=20-49

https://go.dev/play/p/E9Dx1YBzPiN

var numErr *strconv.NumError if errors.As(err, &numErr) { fmt.Printf("invalid %s PS value (%s): %w", key, numErr.Num, numErr.Err) }

Which would be something like;

invalid total PSI value (some invalid value): invalid syntax

strconv.ParseUint is not very relevant to the user.

Indeed it isn't, but at least it says which function returned the error, and any developer (and some advanced users) will deduce that an unsigned integer was expected.

Generally speaking, to make this error more user-friendly, the error text should say something like "expected unsigned integer value, got %q: %w", and the best place to fix this is the actual strconv package.

In this case, though, we're parsing kernel-generated data (and we actually check that it comes from the kernel, by checking the filesystem type in cgroups.OpenFile), and so the chance of getting a non-number here (and thus receive an error) is close to 0 here.

Yeah, it really was just me rambling a bit, having dealt too many times with users "I have the same error" (because the strconv.ParseUint .... didn't resonate with them, and they thought it was a bug, not recognising "invalid input here".

Probably something for a rainy afternoon to look if I can come with some nice approach for these. 😄

libcontainer/cgroups/fscommon/rdma.go

libcontainer/cgroups/fscommon/utils.go

thaJeztah · 2024-12-23T14:12:55Z

libcontainer/cgroups/fscommon/utils.go

-		arr := strings.Split(line, " ")
-		if len(arr) == 2 && arr[0] == key {
-			val, err := ParseUint(arr[1], 10, 64)
+		k, v, ok := strings.Cut(line, " ")
+		if ok && k == key {


Not sure if relevant (and probably new code is more correct?); previously, we would ignore cases with more than 2 fields. I.e., key foo bar would be ignored; in the new code, we'd try to parse foo bar as value.

That said; see my other comment; this would potentially be an issue if we try to parse something that may currently have 2 cols, and in future 3 cols; not sure if that should error or silently ignore

Yes, to me the new behavior is also more correct, and if suddenly a third column appears, it means our parsing code is not adequate and we should fix it.

Not relevant to your comment, but by looking at the code, it might be further improved to do something like this:

key += " " for _, line := range lines { if v, ok := strings.CutPrefix(line, key); ok { val, err := ParseUint(v, 10, 64) ... }

to avoid allocating memory for k.

by looking at the code, it might be further improved

Added a separate commit.

thaJeztah · 2024-12-23T14:26:15Z

exec.go

+			// No controller: prefix.
 			if len(args) != 1 {
 				return nil, fmt.Errorf("invalid --cgroup argument: %s (missing <controller>: prefix)", c)
 			}
 			paths[""] = c


Was trying to somewhat grasp what this case is for;

if zero args are passed, we return early (start of function)

if 1 arg is passed, it MUST have a controller prefix

But if 2 (or more) args are passed, then we don't have to?

So;

# valid: single arg --cgroup a,b:/hello-world # valid: single arg without controller --cgroup /hello-world # invalid: multiple args, one without controller --cgroup a,b:/hello-world --cgroup /hello-world --cgroup /hello-world --cgroup a,b:/hello-world

So, if no controller is passed, it's for everything which is only allowed if no other (per-controller) paths are specified, correct? (so not allowed to be combined).

☝️ Perhaps that's something we should capture in a comment on the function

Also;

perhaps we need to change the ok check to also check if controller is not empty, or is ,,,,,:/hello-world something that should be considered valid?

are duplicate controllers valid? They currently overwrite previous values (foo,foo,foo:/bar, foo:/baz)

if 1 arg is passed, it MUST have a controller prefix

Looks like you got it wrong. The correct version is:

if 1 arg is passed, it can be with or without controller prefix;

if more than 1 arg is passed, they all MUST have a controller prefix.

Specifying more than 1 argument without controller prefix doesn't make sense (i.e. invalid input), and this is what the code is capturing.

In short, a version with controllers is for cgroup v1, as you can (and usually have to) specify different paths for different controllers. A version without controllers is for cgroup v2. This is described in runc-exec(8) man page.

Your other comments (about more comments, allowing empty values for controllers, and allowing to specify duplicated controllers) are all valid, and are addressed by the additional commit (second one in this PR).

kolyshkin · 2025-01-06T23:07:41Z

@lifubang @thaJeztah addressed all your comments, please take another look.

kolyshkin · 2025-01-07T06:35:45Z

Added a few commits (at the end) which uses strings.CutPrefix where appropriate. Those are easier to review as they don't change functionality, just make the code more readable.

kolyshkin · 2025-01-07T06:40:02Z

libcontainer/configs/validate/rootless.go

@@ -65,9 +65,9 @@ func rootlessEUIDMount(config *configs.Config) error {
 				if _, err := config.HostUID(uid); err != nil {
 					return fmt.Errorf("cannot specify uid=%d mount option for rootless container: %w", uid, err)
 				}
+				continue


This is added so when we've found uid=... in opt, we don't also try it for gid= but go to the next option.

Alternative here could be to else if str, ok := strings.CutPrefix(opt, "gid="); ok {, to make it clearer each iteration handles either one or the other.

Yes, my original commit had if / else if but it results in a larger, harder to review commit, so I switched to continue.

Also note that continue (or else) is optional here, as missing it will not result is a bug, but merely in a very slight performance degradation.

@thaJeztah ended up with one more commit changing this to if/else (and other minor nits).

thaJeztah

LGTM, thanks!!

exec.go

+			// No controller: prefix.
 			if len(args) != 1 {
 				return nil, fmt.Errorf("invalid --cgroup argument: %s (missing <controller>: prefix)", c)
 			}
 			paths[""] = c


libcontainer/cgroups/fs2/defaultpath.go

thaJeztah · 2025-01-10T20:38:21Z

libcontainer/cgroups/fs2/psi.go

 		case "avg10":
 			pv = &data.Avg10
 		case "avg60":
 			pv = &data.Avg60
 		case "avg300":
 			pv = &data.Avg300
 		case "total":
-			v, err := strconv.ParseUint(kv[1], 10, 64)
+			v, err := strconv.ParseUint(val, 10, 64)


Yeah, it really was just me rambling a bit, having dealt too many times with users "I have the same error" (because the strconv.ParseUint .... didn't resonate with them, and they thought it was a bug, not recognising "invalid input here".

Probably something for a rainy afternoon to look if I can come with some nice approach for these. 😄

libcontainer/cgroups/fscommon/rdma.go

libcontainer/cgroups/fscommon/utils.go

thaJeztah · 2025-01-10T21:02:03Z

libcontainer/configs/validate/rootless.go

@@ -65,9 +65,9 @@ func rootlessEUIDMount(config *configs.Config) error {
 				if _, err := config.HostUID(uid); err != nil {
 					return fmt.Errorf("cannot specify uid=%d mount option for rootless container: %w", uid, err)
 				}
+				continue


Alternative here could be to else if str, ok := strings.CutPrefix(opt, "gid="); ok {, to make it clearer each iteration handles either one or the other.

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). This part of code is covered by tests in tests/integration/exec.bats. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

1. Document the function. 2. Add sanity checks for empty and repeated controllers. Reported-by: Sebastiaan van Stijn <[email protected]> Signed-off-by: Kir Kolyshkin <[email protected]>

Nowadays strings.Fields are as fast as strings.SplitN so remove TODO. Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). This also drops the check for extra dash (we're unlikely to get it from the kernel anyway). While at it, rename min/max -> from/to to avoid collision with Go min/max builtins. This code is tested by TestCPUSetStats* tests. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

Remove extra global constants that are only used in a single place and make it harder to read the code. Rename nanosecondsInSecond -> nsInSec. This code is tested by unit tests. Signed-off-by: Kir Kolyshkin <[email protected]>

For cgroup v2, we always expect /proc/$PID/cgroup contents like this: > 0::/user.slice/user-1000.slice/[email protected]/app.slice/vte-spawn-f71c3fb8-519d-4e2d-b13e-9252594b1e05.scope So, it does not make sense to parse it using strings.Split, we can just cut the prefix and return the rest. Code tested by TestParseCgroupFromReader. Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). The code is tested by testCgroupResourcesUnified. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). This code is tested by TestStatCPUPSI. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). Also, use switch in parseRdmaKV. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.CutPrefix (added in Go 1.20, see [1]) results in faster and cleaner code with less allocations (as the code only allocates memory for the value, and does it once). While at it, improve the function documentation. [1]: golang/go#42537 Signed-off-by: Kir Kolyshkin <[email protected]>

It makes sense to report an error if a key or a value is empty, as we don't expect anything like this. Reported-by: Sebastiaan van Stijn <[email protected]> Signed-off-by: Kir Kolyshkin <[email protected]>

1. GetCgroupParamUint: drop strings.TrimSpace since it was already done by GetCgroupParamString. 2. GetCgroupParamInt: use GetCgroupParamString, drop strings.TrimSpace. Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.HasPrefix with strings.TrimPrefix results in doing the same thing (checking if prefix exists) twice. In this case, using strings.TrimPrefix right away is sufficient. Signed-off-by: Kir Kolyshkin <[email protected]>

Using strings.CutPrefix (available since Go 1.20) instead of strings.HasPrefix and/or strings.TrimPrefix makes the code a tad more straightforward. No functional change. Signed-off-by: Kir Kolyshkin <[email protected]>

1. Avoid splitting mount data into []string if it does not contain options we're interested in. This should result in slightly less garbage to collect. 2. Use if / else if instead of continue, to make it clearer that we're processing one option at a time. 3. Print the whole option as a sting in an error message; practically this should not have any effect, it's just simpler. Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin force-pushed the strings-cut branch from 92d3c96 to bf53e34 Compare October 23, 2024 06:59

kolyshkin added the kind/refactor refactoring label Oct 23, 2024

kolyshkin force-pushed the strings-cut branch from bf53e34 to aeb74e8 Compare October 23, 2024 19:01

kolyshkin requested review from dqminh and rata and removed request for dqminh November 2, 2024 00:37

kolyshkin force-pushed the strings-cut branch from aeb74e8 to 732da9a Compare November 6, 2024 20:53

kolyshkin force-pushed the strings-cut branch from 732da9a to ae23108 Compare November 7, 2024 20:59

kolyshkin force-pushed the strings-cut branch 2 times, most recently from 819df22 to 62a4183 Compare December 5, 2024 00:04

kolyshkin added this to the 1.3.0 milestone Dec 5, 2024

kolyshkin force-pushed the strings-cut branch from 62a4183 to 37fdfdd Compare December 19, 2024 23:32

lifubang reviewed Dec 20, 2024

View reviewed changes

kolyshkin force-pushed the strings-cut branch from 37fdfdd to 247d92a Compare December 21, 2024 01:01

kolyshkin requested a review from lifubang December 23, 2024 01:46

kolyshkin force-pushed the strings-cut branch from 247d92a to cddca40 Compare December 23, 2024 01:46

thaJeztah reviewed Dec 23, 2024

View reviewed changes

kolyshkin force-pushed the strings-cut branch from cddca40 to 9a5cf6e Compare January 6, 2025 22:11

kolyshkin requested a review from thaJeztah January 6, 2025 22:15

kolyshkin force-pushed the strings-cut branch from 9a5cf6e to 17845ca Compare January 6, 2025 23:04

kolyshkin force-pushed the strings-cut branch from 17845ca to f94cf87 Compare January 7, 2025 04:24

kolyshkin changed the title ~~Use strings.Cut where possible~~ Use strings.Cut and strings.CutPrefix where possible Jan 7, 2025

kolyshkin commented Jan 7, 2025

View reviewed changes

kolyshkin force-pushed the strings-cut branch from f94cf87 to dbac61d Compare January 10, 2025 20:25

thaJeztah approved these changes Jan 10, 2025

View reviewed changes

kolyshkin added 16 commits January 10, 2025 15:50

exec: improve getSubCgroupPaths

93e16bc

1. Document the function. 2. Add sanity checks for empty and repeated controllers. Reported-by: Sebastiaan van Stijn <[email protected]> Signed-off-by: Kir Kolyshkin <[email protected]>

libct/cg/fs: getPercpuUsage: rm TODO

314d28f

Nowadays strings.Fields are as fast as strings.SplitN so remove TODO. Signed-off-by: Kir Kolyshkin <[email protected]>

libcontainer/cgroups/fs: some refactoring

e12c1f4

Remove extra global constants that are only used in a single place and make it harder to read the code. Rename nanosecondsInSecond -> nsInSec. This code is tested by unit tests. Signed-off-by: Kir Kolyshkin <[email protected]>

libct/cg/fscommon: use strings.Cut in RDMA parser

0a2862b

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). Also, use switch in parseRdmaKV. [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

libct/cg/fscommon: ParseKeyValue: use strings.Cut

5a77f43

Using strings.Cut (added in Go 1.18, see [1]) results in faster and cleaner code with less allocations (as we're not using a slice). [1]: golang/go#46336 Signed-off-by: Kir Kolyshkin <[email protected]>

libct/cg/fscommon: ParseKeyValue: stricter check

fae6ea0

It makes sense to report an error if a key or a value is empty, as we don't expect anything like this. Reported-by: Sebastiaan van Stijn <[email protected]> Signed-off-by: Kir Kolyshkin <[email protected]>

libct/cg/fscommon: GetCgroupParam*: unify

7ed63fe

1. GetCgroupParamUint: drop strings.TrimSpace since it was already done by GetCgroupParamString. 2. GetCgroupParamInt: use GetCgroupParamString, drop strings.TrimSpace. Signed-off-by: Kir Kolyshkin <[email protected]>

libct/utils: stripRoot: rm useless HasPrefix

480792b

Using strings.HasPrefix with strings.TrimPrefix results in doing the same thing (checking if prefix exists) twice. In this case, using strings.TrimPrefix right away is sufficient. Signed-off-by: Kir Kolyshkin <[email protected]>

libct: use strings.CutPrefix where possible

05f53d6

Using strings.CutPrefix (available since Go 1.20) instead of strings.HasPrefix and/or strings.TrimPrefix makes the code a tad more straightforward. No functional change. Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin force-pushed the strings-cut branch from dbac61d to 8ecb278 Compare January 10, 2025 23:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use strings.Cut and strings.CutPrefix where possible #4470

Use strings.Cut and strings.CutPrefix where possible #4470

kolyshkin commented Oct 23, 2024 •

edited

Loading

rata commented Nov 5, 2024 •

edited

Loading

kolyshkin commented Nov 6, 2024 •

edited

Loading

kolyshkin commented Dec 5, 2024

lifubang Dec 20, 2024 •

edited

Loading

kolyshkin Dec 21, 2024

thaJeztah Dec 23, 2024

thaJeztah Dec 23, 2024

kolyshkin Jan 6, 2025

thaJeztah Jan 10, 2025

thaJeztah Dec 23, 2024

kolyshkin Jan 6, 2025

kolyshkin Jan 6, 2025

thaJeztah Dec 23, 2024

kolyshkin Jan 6, 2025

This comment was marked as resolved.

kolyshkin commented Jan 6, 2025

kolyshkin commented Jan 7, 2025

kolyshkin Jan 7, 2025

thaJeztah Jan 10, 2025

kolyshkin Jan 10, 2025

kolyshkin Jan 10, 2025

thaJeztah left a comment

This comment was marked as resolved.

thaJeztah Jan 10, 2025

thaJeztah Jan 10, 2025

Use strings.Cut and strings.CutPrefix where possible #4470

Are you sure you want to change the base?

Use strings.Cut and strings.CutPrefix where possible #4470

Conversation

kolyshkin commented Oct 23, 2024 • edited Loading

rata commented Nov 5, 2024 • edited Loading

kolyshkin commented Nov 6, 2024 • edited Loading

kolyshkin commented Dec 5, 2024

lifubang Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

kolyshkin commented Jan 6, 2025

kolyshkin commented Jan 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thaJeztah left a comment

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kolyshkin commented Oct 23, 2024 •

edited

Loading

rata commented Nov 5, 2024 •

edited

Loading

kolyshkin commented Nov 6, 2024 •

edited

Loading

lifubang Dec 20, 2024 •

edited

Loading