Fine-grained Control Over Caching for Deployment Targets in NX

[aramfe]

Description

We currently use NX with NX Cloud and DTE (Deployment Task Execution) for our CI/CD pipeline. To ensure that deployment-related targets are not cached between pipeline runs, we use dynamic inputs for these targets. This approach works well for NX Cloud and DTE, but presents challenges when trying to execute locally.

Key points:

• Deployment targets use dynamic inputs to prevent caching across all runs, but not in a current run/retry context
• These inputs rely on environment variables like GITHUB_RUN_NUMBER and GITHUB_RUN_ATTEMPT
• Local execution requires manual setting of these environment variables to ensure they are not cached as well, which requires manually setting GITHUB_RUN_NUMBER and GITHUB_RUN_ATTEMPT

Current Behavior

CI/CD Pipeline:

• Targets use dynamic inputs for deployment-related tasks
• Inputs are generated dynamically based on pipeline run number and target

Local Execution:

• Requires manual setting of environment variables (e.g., GITHUB_RUN_NUMBER, GITHUB_RUN_ATTEMPT) which is not error prone due to users forgetting to set it, etc.
• Without proper setup, local runs may cache incorrectly

Proposed Solution

Implement a configuration option for deployment targets that allows them to be executed locally without being affected by cached inputs. Specifically:

1. Add a new option to target configurations:

   • "cache_if_ci": true

2. When set to true:

   • Use cache with inputs only in a CI/CD context, but not locally

3. When set to false:

   • Execution behaves normally, depending on wether or not cache is set to true

4. When both cache and cache_if_ci set to true:

   • Ignore cache_if_ci and behave normally

Example Configuration

{
  "deploy-some": {
    "executor": "nx:run-commands",
    "inputs": ["default", "^default", "sharedGlobals", "deployment"],
    "options": {
      "commands": ["scripts/some-deploy.sh"]
    },
    "cwd": "{projectRoot}", 
    "cache_if_ci": true
  }
}

Benefits

1. Simplifies local development workflow
2. Allows logically different behavior across CI/CD and local environments
3. Reduces manual setup required for local testing

Technical Considerations

• Implement logic to detect whether the build is running in a CI/CD environment (mostly set by the CI/CD services automatically)
• Ensure that when cache_if_ci is true, the target behaves identically in both local and CI/CD environments
• Maintain backwards compatibility for existing projects

Why Current Approach is Suboptimal

Using --skip-nx-cache for deployment targets is not ideal because:

1. It would be applied to every single dependency, which is not the intended behavior
2. Sometimes builds and dependent tasks of a deploy target should re-use cached results
3. Completely disabling caching for deployment targets contradicts the principle of efficient task execution that Nx provides

Next Steps

Please review this proposal and let us know if you need any clarification or have suggestions for implementation.

[aramfe]

I'm fully aware that NX's normal use case is usually for Continuous Integration only (builds, linting, testing, etc.).

However, NX fully utilizes its capabilities if also used for deployments - due to the NX replay functionality (which makes total sense for having all logs/reports in one place), one needs to make these normally non-cacheable targets sadlly also cacheable as well and with that replayable for the main runner in nx cloud.

I'm pretty sure a more ideal solution would be something where there is no real caching from the user side required, and where this is entirely handled by NX itself.

Ideally, one wouldn't even need to define any kind of caching for deployment-related targets (e.g., CloudFormation deployments, pushing docker images, etc.), but could simply enable some kind of option like:

  "deploy-some": {
    "executor": "nx:run-commands",
    "options": {
      "commands": ["scripts/some-deploy.sh"]
    },
    "cwd": "{projectRoot}", 
    "force_dte": true 
  }
}

Because that's basically how it would be normally expected - no caches, yet runnable via DTE on NX cloud. The only reason we define caches at all in targets like that, is simply to make them replayable and runnable with DTE enabled.