Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

treewide: don’t set ulimit -n in builds #351783

Draft
wants to merge 2 commits into
base: staging
Choose a base branch
from

Conversation

emilazy
Copy link
Member

@emilazy emilazy commented Oct 28, 2024

This avoids having to work around the incredibly low Darwin default in a bunch of builds. The Nix daemon is meant to get a higher limit from launchd, but it seems to not work or not be passed down to builds for whatever reason. For now, this is a simple and expedient fix for a recurring problem.

This was a problem on the NixOS infra that was fixed so now these are redundant even without the stdenv patch.

See:

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@emilazy
Copy link
Member Author

emilazy commented Oct 28, 2024

It turns out that this was actually caused by a Hydra misconfiguration that affected Linux too (albeit to a lesser degree). We’re in the process of testing a fix and if it all goes well then we can probably do the tree‐wide portion of this without the stdenv change.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 10, 2024
Copy link
Contributor

@paparodeo paparodeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good sans the change to setup.sh

Comment on lines 536 to 545
# Increase the soft file descriptor limit if it’s too low. This is
# mostly for the benefit of Darwin, where the default is a positively
# anaemic 256 FDs by default. We use 10240 because it’s the default
# value of `kern.maxfilesperproc` on macOS 10.12, but this can be
# increased when the minimum macOS version is bumped.
if [[ "$(ulimit -n)" < 10240 ]]; then
ulimit -S -n 10240 || true
fi


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Increase the soft file descriptor limit if it’s too low. This is
# mostly for the benefit of Darwin, where the default is a positively
# anaemic 256 FDs by default. We use 10240 because it’s the default
# value of `kern.maxfilesperproc` on macOS 10.12, but this can be
# increased when the minimum macOS version is bumped.
if [[ "$(ulimit -n)" < 10240 ]]; then
ulimit -S -n 10240 || true
fi

This is now handled in the daemon’s environment.
@github-actions github-actions bot removed the 6.topic: stdenv Standard environment label Nov 25, 2024
@emilazy emilazy changed the title stdenv: increase ulimit -n if it’s too low treewide: don’t set ulimit -n in builds Nov 25, 2024
@emilazy emilazy marked this pull request as ready for review November 25, 2024 22:27
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple as.

@emilazy
Copy link
Member Author

emilazy commented Nov 25, 2024

I’ve dropped the stdenv commit. This is still likely to make things break more under sudo nix build … and such? But arguably that’s a footgun, especially if you don’t configure good limits yourself. Hydra having low limits would have kept it mostly working in the past, but all bets are off now.

This is handled in the daemon’s environment.
@emilazy
Copy link
Member Author

emilazy commented Nov 25, 2024

Fixed the Darwin glibc cross patch probably. Embarrassing mistake. Someone should verify the build.

@paparodeo
Copy link
Contributor

paparodeo commented Nov 25, 2024

Fixed the Darwin glibc cross patch probably. Embarrassing mistake. Someone should verify the build.

missed that in reviewing it too. diffs of diffs are the worst. and it also seems like the formatter switches rules on where to put the '' and indention based on if there is a + or not, which is cluttering the diffs too.

edit: tho hide whitespace takes care of formatter sins.

@philiptaron
Copy link
Contributor

I also missed the glibc thing when reading through. 🫣

@philiptaron philiptaron removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 25, 2024
@emilazy emilazy marked this pull request as draft November 26, 2024 11:53
@emilazy
Copy link
Member Author

emilazy commented Nov 26, 2024

The patch needs fixing.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants