Add -pr/--priority/KOMETA_PRIORITY to allow Kometa to run at lower/higher priorities

[YozoraXCII]

Description

Credits to @planetrocky for the initial Pull Request.

This change allows the user to specify the priority that Kometa run at on Windows/Linux/Mac (and should work in Docker too).

The options available are:
25: Below Normal
50: Normal (default)
75: Above Normal
90: High
100: Realtime

Issues Fixed or Closed

Closes #2304 (supersedes)

Type of Change

• New feature (non-breaking change which adds functionality)

Checklist

• Updated Documentation to reflect changes
• Updated the CHANGELOG with the changes

[YozoraXCII]

I would greatly appreciate any testing on this, particularly the KOMETA_PRIORITY=XX as I haven't been able to test this yet.

Docker image is kometateam/kometa:run-nicely

[planetrocky]

@YozoraXCII just carried out a visual code-review.

I can’t see low_priority() being called anywhere?

Originally I only added lowering the priority; otherwise in Docker you have to add —cap-add SYS_NICE to the Docker container if you want to use a negative nice value (that’s a privileged operation in Linux and requires root).
I was also concerned that users might just run their Docker with --privileged rather than the fine-grained permissions with —cap-add.

Should the definition be changed to reflect it is now changing Kometa’s run process priority, not just lowering/nice-ing the process.

[YozoraXCII]

@YozoraXCII just carried out a visual code-review.

I can’t see low_priority() being called anywhere?

Originally I only added lowering the priority; otherwise in Docker you have to add —cap-add SYS_NICE to the Docker container if you want to use a negative nice value (that’s a privileged operation in Linux and requires root). I was also concerned that users might just run their Docker with --privileged rather than the fine-grained permissions with —cap-add.

Should the definition be changed to reflect it is now changing Kometa’s run process priority, not just lowering/nice-ing the process.

Let me take this away and think about how to handle it. I may need to reduce the scope if we're going to potentially run into privilege issues.

[planetrocky]

@YozoraXCII just carried out a visual code-review.
I can’t see low_priority() being called anywhere?
Originally I only added lowering the priority; otherwise in Docker you have to add —cap-add SYS_NICE to the Docker container if you want to use a negative nice value (that’s a privileged operation in Linux and requires root). I was also concerned that users might just run their Docker with --privileged rather than the fine-grained permissions with —cap-add.
Should the definition be changed to reflect it is now changing Kometa’s run process priority, not just lowering/nice-ing the process.

Let me take this away and think about how to handle it. I may need to reduce the scope if we're going to potentially run into privilege issues.

I wrote a long reply about this. However I a got BSOD on my Windows machine before hitting comment :(

[planetrocky]

@YozoraXCII just carried out a visual code-review.
I can’t see low_priority() being called anywhere?
Originally I only added lowering the priority; otherwise in Docker you have to add —cap-add SYS_NICE to the Docker container if you want to use a negative nice value (that’s a privileged operation in Linux and requires root). I was also concerned that users might just run their Docker with --privileged rather than the fine-grained permissions with —cap-add.
Should the definition be changed to reflect it is now changing Kometa’s run process priority, not just lowering/nice-ing the process.

Let me take this away and think about how to handle it. I may need to reduce the scope if we're going to potentially run into privilege issues.

In the previous PR we discussed this, there was a question about adding the ability to also increase the process priority. I was reluctant for a few reasons:

In the previous PR, we discussed adding the ability to also increase the process priority. I was reluctant for a few reasons:

1. Long-running processing jobs such as Kometa are generally best at normal/default scheduling priority. Lowering Kometa's process priority benefits the system by improving responsiveness, user interactivity, and reducing the impact on system processes that run at higher priority. Setting Kometa to match system-level high-priority processes is not optimal. Kometa can be a long-running job with heavy I/O and would contend for scheduling time with system-level jobs. High-priority tasks often take resources exclusively and should release them as soon as possible. Contention at this scheduling level could degrade performance and perhaps stability.

   Often in multiprocessing, there will be synchronization or locking of shared resources. If a lock, mutex, or semaphore is taken, anything else dependent on that will block. The design might run blocking aspects at a higher priority to reduce the impact. Setting Kometa to a normal/default user priority allows system high-priority processes to take precedence in scheduling and prevents competition for resources with critical system operations.

   This is why increasing a process priority is always a privileged operation. Lowering Kometa's process priority below normal (adding a nice value to the base priority) enables interactive processes or other short-running jobs to take precedence. In my use case, I run a Linux Docker image for Kometa updates, typically on my desktop. When I am making many changes, adding things, and testing new overlays, I often have Kometa running at a lower priority for many hours every evening. This is precisely what the Linux nice mechanism was designed for. Modern OS schedulers are adept at maximizing resource utilization.

2. Under Linux and Docker, an ordinary user may lower the process priority by setting a nice value, which is encouraged. However, setting a negative nice value (high priority) requires UID=0 (root) privileges. Docker provides --cap-add to permit fine-grained control over which privileged operations a non-root Docker instance may execute. For this case, the appropriate capability would be --cap-add SYS_NICE.

   Some users, however, might run the Docker instance as root (--privileged), which is unnecessary and more dangerous. Some Docker GUIs offer a simple checkbox to run as privileged, which undermines container isolation. Docker themselves describe this mode as follows:

   This flag exists to allow special use-cases, like running Docker within Docker.

Escalate container privileges (--privileged)

Escalate container privileges

The --privileged flag gives the following capabilities to a container:

• Enables all Linux kernel capabilities
• Disables the default seccomp profile
• Disables the default AppArmor profile
• Disables the SELinux process label
• Grants access to all host devices
• Makes /sys read-write
• Makes cgroups mounts read-write

In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker.

⚠️Warning

Use the --privileged flag with caution. A container with --privileged is not a securely sandboxed process. Containers in this mode can get a root shell on the host and take control over the system.

For most use cases, this flag should not be the preferred solution. If your container requires escalated privileges, you should prefer to explicitly grant the necessary permissions, for example by adding individual kernel capabilities with --cap-add.

For more information, see Runtime privilege and Linux capabilities

[YozoraXCII]

@YozoraXCII just carried out a visual code-review.
I can’t see low_priority() being called anywhere?
Originally I only added lowering the priority; otherwise in Docker you have to add —cap-add SYS_NICE to the Docker container if you want to use a negative nice value (that’s a privileged operation in Linux and requires root). I was also concerned that users might just run their Docker with --privileged rather than the fine-grained permissions with —cap-add.
Should the definition be changed to reflect it is now changing Kometa’s run process priority, not just lowering/nice-ing the process.

Let me take this away and think about how to handle it. I may need to reduce the scope if we're going to potentially run into privilege issues.

In the previous PR we discussed this, there was a question about adding the ability to also increase the process priority. I was reluctant for a few reasons:

In the previous PR, we discussed adding the ability to also increase the process priority. I was reluctant for a few reasons:

1. Long-running processing jobs such as Kometa are generally best at normal/default scheduling priority. Lowering Kometa's process priority benefits the system by improving responsiveness, user interactivity, and reducing the impact on system processes that run at higher priority. Setting Kometa to match system-level high-priority processes is not optimal. Kometa can be a long-running job with heavy I/O and would contend for scheduling time with system-level jobs. High-priority tasks often take resources exclusively and should release them as soon as possible. Contention at this scheduling level could degrade performance and perhaps stability.
   Often in multiprocessing, there will be synchronization or locking of shared resources. If a lock, mutex, or semaphore is taken, anything else dependent on that will block. The design might run blocking aspects at a higher priority to reduce the impact. Setting Kometa to a normal/default user priority allows system high-priority processes to take precedence in scheduling and prevents competition for resources with critical system operations.
   This is why increasing a process priority is always a privileged operation. Lowering Kometa's process priority below normal (adding a nice value to the base priority) enables interactive processes or other short-running jobs to take precedence. In my use case, I run a Linux Docker image for Kometa updates, typically on my desktop. When I am making many changes, adding things, and testing new overlays, I often have Kometa running at a lower priority for many hours every evening. This is precisely what the Linux nice mechanism was designed for. Modern OS schedulers are adept at maximizing resource utilization.
2. Under Linux and Docker, an ordinary user may lower the process priority by setting a nice value, which is encouraged. However, setting a negative nice value (high priority) requires UID=0 (root) privileges. Docker provides --cap-add to permit fine-grained control over which privileged operations a non-root Docker instance may execute. For this case, the appropriate capability would be --cap-add SYS_NICE.
   Some users, however, might run the Docker instance as root (--privileged), which is unnecessary and more dangerous. Some Docker GUIs offer a simple checkbox to run as privileged, which undermines container isolation. Docker themselves describe this mode as follows:

   This flag exists to allow special use-cases, like running Docker within Docker.

Escalate container privileges (--privileged)

Escalate container privileges

The --privileged flag gives the following capabilities to a container:

• Enables all Linux kernel capabilities
• Disables the default seccomp profile
• Disables the default AppArmor profile
• Disables the SELinux process label
• Grants access to all host devices
• Makes /sys read-write
• Makes cgroups mounts read-write

In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker.

⚠️Warning
Use the --privileged flag with caution. A container with --privileged is not a securely sandboxed process. Containers in this mode can get a root shell on the host and take control over the system.
For most use cases, this flag should not be the preferred solution. If your container requires escalated privileges, you should prefer to explicitly grant the necessary permissions, for example by adding individual kernel capabilities with --cap-add.
For more information, see Runtime privilege and Linux capabilities

Not ignoring this, just haven't had the headspace to read and contemplate all this - it's a bit above my knowledge level if I'm honest lol.