process: add execve #56496

ShogunPanda · 2025-01-07T06:26:52Z

This PR adds a new process.execve method (absolutely willing to change the name if you want), which is a wrapper for the execve UNIX function.

The function will never return and will swap the current process with a new one.
All memory and system resources are automatically collected from execve, except for std{in,out,err}.

The primary use of this function is in shell scripts to allow to setup proper logics and then spawn another command.

nodejs-github-bot · 2025-01-07T06:26:57Z

Review requested:

@nodejs/startup

nodejs-github-bot · 2025-01-07T06:34:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64381/

ljharb · 2025-01-07T06:35:37Z

What does it do on non-Unix systems?

ShogunPanda · 2025-01-07T07:18:59Z

@ljharb By non Unix we only mean Windows, isn't it? If that's the case, Windows also supports execve via _execve (doc here) so we should be good to go.

Once the CI ends I'll see which platform needs special assistance.

codecov · 2025-01-07T07:41:54Z

Codecov Report

Attention: Patch coverage is 80.73394% with 21 lines in your changes missing coverage. Please review.

Project coverage is 89.18%. Comparing base (afafee2) to head (c65b286).
Report is 56 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_process_methods.cc	63.79%	18 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #56496      +/-   ##
==========================================
+ Coverage   88.53%   89.18%   +0.64%     
==========================================
  Files         657      662       +5     
  Lines      190761   191868    +1107     
  Branches    36616    36908     +292     
==========================================
+ Hits       168899   171111    +2212     
+ Misses      15048    13631    -1417     
- Partials     6814     7126     +312

Files with missing lines	Coverage Δ
lib/internal/bootstrap/node.js	`99.57% <100.00%> (+<0.01%)`	⬆️
lib/internal/process/per_thread.js	`99.39% <100.00%> (+0.07%)`	⬆️
src/node_errors.h	`85.00% <ø> (ø)`
src/node_process_methods.cc	`87.24% <63.79%> (+1.41%)`	⬆️

... and 120 files with indirect coverage changes

targos · 2025-01-07T07:45:44Z

On Windows:

D:\a\node\node\src\node_process_methods.cc(544,26): error C2065: 'F_GETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(544,17): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(546,17): error C2065: 'FD_CLOEXEC': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,16): error C2065: 'F_SETFD': undeclared identifier [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')
  
D:\a\node\node\src\node_process_methods.cc(547,7): error C3861: 'fcntl': identifier not found [D:\a\node\node\libnode.vcxproj]
  (compiling source file '/src/node_process_methods.cc')

targos

To fix https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/40120/

test/parallel/test-process-replace.js

test/parallel/test-process-replace-fail.js

test/parallel/test-process-replace-socket.js

src/node_process_methods.cc

ardinugrxha · 2025-01-07T08:02:50Z

src/node_process_methods.cc

+    node::Utf8Value pathname_string(env->isolate(), args[0].As<String>());
+
+    argv = new char*[2];
+    argv[0] = strdup(*pathname_string);


we need check not null also in here, maybe(?)

Good spot. Added.

I think we should use std::memcpy or a similar modern C++ function other than this.

mcollina

lgtm

test/parallel/test-process-replace-fail.js

ShogunPanda · 2025-01-07T14:04:11Z

@ljharb I was totally wrong. Despite of naming and similar signatures, the function _execve only creates a new process without replacing the old one.

I'll disable this function on Windows.

nodejs-github-bot · 2025-01-07T15:36:12Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64387/

doc/api/process.md

addaleax

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

This requires integration with the permissions API or is otherwise an immediate security hole.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

doc/api/errors.md

addaleax · 2025-01-07T17:16:37Z

doc/api/process.md

+resources from the current process are preserved, except for the standard input,
+standard output and standard error file descriptor.
+
+All other resources are discarded by system when the processes are swapped.


What if this isn't the a desirable behavior in a given situation?

Can you please expand this? What do you mean?

If there's any point in adding execve(), it's the customizability that the method brings with it. Leaving file descriptors open and/or redirecting them intentionally can be part of that -- just like you could in a bash script do exec bash 0<&4 1>&4 to create a shell that reads and writes to e.g. a pre-opened socket or something along those lines.

(With a similar reasoning, you could also allow users to actually specify argv[0] with a value they control -- it doesn't need to equal the filename that's being executed)

src/node_process_methods.cc

ljharb · 2025-01-07T18:09:59Z

Why would we want to add a function that can't work on all tier 1 platforms?

jasnell · 2025-01-07T18:24:37Z

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

ljharb · 2025-01-07T18:25:44Z

Gotcha, i wasn't aware of that.

jasnell · 2025-01-07T18:29:57Z

I don't consider it to be ideal. I would have preferred a pattern like process.posix.getegid() similar to what we have with path.posix but these predate my involvement so they are what they are.

jasnell · 2025-01-07T18:32:10Z

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

lib/internal/process/per_thread.js

src/node_errors.h

src/node_process_methods.cc

ShogunPanda · 2025-01-08T05:29:22Z

Why would we want to add a function that can't work on all tier 1 platforms?

Well, we already have a number of such apis... process.getegid() for instance. There are actually quite a few already on process.

Thanks for involuntary hint. :)
I updated the code with #ifdef __POSIX__ and the docs to use the same wording already used there.

ShogunPanda · 2025-01-08T06:03:07Z

@addaleax

If this is to be added, I'd absolutely recommend referring to it by a standard name such as execve(), or otherwise something that makes it clear that it spawns a new process.

As requested, I've renamed it to execve. I didn't like the original name either :)

This requires integration with the permissions API or is otherwise an immediate security hole.

I just added integration with the permission API. It will require --allow-child-process.

Overall I'd recommend not adding this though, unless there's a concrete reason to believe that it fills a significant gap that the existing child_process API doesn't cover.

In the shell scripting context, there is no way to create a new process which would replace the current one. Think about using Node.js to build a complex command to run. After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

ShogunPanda · 2025-01-08T06:11:00Z

@jasnell

+1 on @addaleax's alternative name suggestion. I'm fine with adding this so long as the cleanup logic/expectations are clearly documented.

I added two tests which check that:

Test what happens if we leave a socket open and we try to reopen in the swapped process: the operation succeeds, which means the fd is destroyed. I also tried to install a on('close') but it is not invoked.
Test what happens if there is a process.on('exit') on the original process. It does not get invoked, which means that the system call immediately swap the programs without running any cleanup logic.

I've added this to the documentation to reflect that. Does it look good now?

addaleax · 2025-01-08T06:32:45Z

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

ShogunPanda · 2025-01-08T07:10:48Z

After it, Node.js was not used anymore but since there was no way to swap the process, you would have to spawn a new process, manage it's stdin/stdout/stderr and so forth. That's why I added execve.

I mean, to be clear, this is still a one-line operation (managing stdio literally just comes down to setting stdio: 'inherit') for the most part. I know where you're coming from but I wouldn't add this just for the sake of adding it.

I kinda agree.
You still would have to handle the exit code to ensure proper broadcasting to the shell.
Also, I'm not sure what would happen with "TUI" (ncurses and similar) programs and so forth.

Is your objection a full block or just an "intent"?

addaleax · 2025-01-08T07:22:29Z

@ShogunPanda Just fyi, have you seen https://www.npmjs.com/package/foreground-child? That also works on Windows 🙂

My "request changes" marker only refers to the C++ memory management here, i.e. this comment specifically (and the integration with the permissions API, of course). I don't intend to block this feature, I've given my opinion but it's pretty clear overall that the Node.js project has a different stance from my own when it comes to 'what should go into core' 🙂

src/node_errors.h

src/node_process_methods.cc

lib/internal/process/per_thread.js

src/node_process_methods.cc

test/parallel/test-process-execve-fail.js

ShogunPanda · 2025-01-11T08:24:46Z

@anonrig @addaleax @jasnell I have vastly refactored the C++ to address Anna's and James' suggestions. Can you please review it again?

Apparently I'm more a C guy than a C++ guy. And last time I officially touched C was several years ago. Lol.

Thanks for the patience here!

src/node_process_methods.cc

jasnell · 2025-01-11T14:46:29Z

src/node_process_methods.cc

+  execve(*executable, argv.data(), envp.data());
+  int error_code = errno;
+
+  THROW_ERR_PROCESS_EXECVE_FAILED(env, error_code);


Given that you're running the RunAtExit(...) here, and given that the expectation is for this process to be replaced, I wonder if it would be more appropriate to crash the process here instead of just throwing (a throw can be caught and ignored and I'm not sure we want that here)

Good point.
I update that. Can you check if I used the proper methods to print the errors before aborting or was it there a better way?

addaleax · 2025-01-13T17:56:19Z

src/node_process_methods.cc

+  // so that the new process will inherit it.
+  if (persist_standard_stream(0) < 0 || persist_standard_stream(1) < 0 ||
+      persist_standard_stream(2) < 0) {
+    env->ThrowErrnoException(errno, "fcntl");


Suggested change

env->ThrowErrnoException(errno, "fcntl");

env->ThrowErrnoException(errno, "fcntl");

return;

addaleax · 2025-01-13T17:56:37Z

src/node_process_methods.cc

+  std::string info = FormatErrorMessage(
+      isolate, context, error_message.c_str(), message, true);
+  FPrintF(stderr, "%s\n", info);
+  ABORT();


Why does this abort instead of throwing?

@addaleax ... see #56496 (comment)

I think this is something we can reasonably debate a bit.

Hm yeah ... I'd argue that this further limits the already fairly limited usefulness of this feature, but I understand where it's coming from 🙂 Just seems odd that this hard crashes in this case, you can't always know for sure that execve() will succeed

Yep, this one is rather tricky. I'm not quite sure what this should do really and absolutely open to alternatives. One suggestion is to mark this new API as experimental, stick with the abort for now, and see if a reasonable alternative emerges before we graduate it but I'm not really certain what the ideal approach here is.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. process Issues and PRs related to the process subsystem. labels Jan 7, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

targos reviewed Jan 7, 2025

View reviewed changes

ardinugrxha reviewed Jan 7, 2025

View reviewed changes

mcollina approved these changes Jan 7, 2025

View reviewed changes

richardlau reviewed Jan 7, 2025

View reviewed changes

test/parallel/test-process-replace-fail.js Outdated Show resolved Hide resolved

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

jasnell reviewed Jan 7, 2025

View reviewed changes

doc/api/process.md Show resolved Hide resolved

addaleax requested changes Jan 7, 2025

View reviewed changes