yarn 3 - amplify push then amplify pull deletes yarn.lock, .yarnrc.yml then errors on building typescript override

[JefStat]

How did you install the Amplify CLI?

npm install -g @aws-amplify/cli

If applicable, what version of Node.js are you using?

16.19.1

Amplify CLI Version

11.0.3

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

amplify pull deletes files required for the prebuild of overrides when running a backend push

Expected behavior

push and pull to be a no-op if there are no changes

Reproduction steps

Steps:

1. AWS_PROFILE=amplify amplify push
2. AWS_PROFILE=amplify amplify pull
   3 of these 4 files have been deleted yarn.lock, .yarnrc.yml, APIGatewayAuthStack.json
   

Project Identifier

729269ed0dfa7238ed8a5b3b8f9c5f9a

Log output

# Put your logs below this line
Pre-pull status:

    Current Environment: dev

 Building resource auth/commandcentral772995b7🛑 Packaging overrides failed.
Command failed with exit code 1: yarn install
Usage Error: The nearest package directory (command-central/amplify/backend) doesn't seem to be part of the project declared in command-central.

Resolution: There may be errors in your overrides file. If so, fix the errors and try again.
Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
✔ Successfully pulled backend environment dev from the cloud.
✅

Session Identifier: 12b3c099-f775-4b33-aaa0-bff61ed621ce

Additional information

• Checkout works without editing the yarn configuration. eg AWS_PROFILE=amplify amplify env checkout dev
• Yarn is install via corepack npm -g i corepack; corepack enable; corepack prepare yarn@stable --activate

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[josefaidt]

Hey @JefStat 👋 thanks for raising this! While we have a separate report filed for APIGatewayAuthStack missing after an amplify pull, I was not able to reproduce the issue where yarn.lock is deleted entirely. Strangely enough I noticed the file was deleted during amplify pull, then modified with updated patch dependencies (likely from yarn install without a frozen lockfile), then returned to its original state. I will try again after installing yarn 2/3

Did amplify pull fail for any reason?

[JefStat]

Did amplify pull fail for any reason?

I'm not clear on how to answer this. The pull works. I assume the push is not including the yarn.lock in the zip. I need to check if there is a force push to inspect why it's not including yarn.lock

Debugging further:

The problem happens here: The backend dir is deleted wiping away the local yarn.lock

initialize-env.ts

    if (context.exeInfo.restoreBackend) {
      fs.removeSync(backendDir);
      fs.copySync(unzippedDir, backendDir);
    }

The restoreBackend flag is set to true because the noOverride flag hasn't been set.

pull-backend.ts
context.exeInfo.restoreBackend = !context.exeInfo.inputParams.amplify.noOverride;

[JefStat]

I've sorted this. TLDR cannot use yarn 3.5.0 in the backend.

I am using 2 yarn versions in the repo via the package manager settings.

amplify/backend/package.json

  "packageManager": "[email protected]",
  "workspaces": [
    "function/myfunctionName/src"
  ]

./package.json

"packageManager": "[email protected]",
"workspaces":[]

Amplify push fails because the backend cannot use yarn 3.5.0 due to this unsupported option in:

legacyBuild.ts

const args = useYarn ? ['--no-bin-links'] : ['install', '--no-bin-links'];

issue ref: #10815

I probably hamstrung myself pushing a backend bundle without the yarn.lock file while iterating over settings trying to get amplify to use the version of yarn I expected for the frontend and backend builds

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[josefaidt]

Hey @JefStat thank you for providing those details! Despite the oddity I'd like to keep this issue open to introduce better handling for yarn 3

[JefStat]

The following documentation will not work with yarn classic either because of the --no-bin-links legacy build option
https://docs.amplify.aws/cli/function/build-options/

In the following script in package.json the tsc -p will fail because there is no longer a tsc in the .bin folder. (which is what led me down this rabbit hole)

"amplify:generateReport": "cd amplify/backend/function/generateReport && tsc -p ./tsconfig.json && cd -"

The alternative which I am using is some akin to:

"amplify:generateReport": "cd amplify/backend/function/generateReport && cd ../../../..  && tsc -p amplify/backend/function/generateReport/tsconfig.json"

(cd - doesn't work in the npm subshell idk thus the chain of ../)

[josefaidt]

Hey @JefStat yes, I am also seeing a few issues deploying with yarn 3, namely the lack of support for --no-bin-links

This issue is also similar to #6382, which is requesting support for additional package managers outside npm and yarn 1.x, namely pnpm.

I'll mark this as a duplicate feature-request to improve the experience here with yarn 3, and address the vague error on amplify push in the meantime

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.