Support PNPM and Others

[shellscape]

Note: If your feature-request is regarding the AWS Amplify Console service, please log it in the
official AWS Amplify Console forum

Is your feature request related to a problem? Please describe.

I suppose you could consider this related to a problem; at present the Amplify CLI is forcing either npm or yarn upon the userbase. Locking users into a vendor is never a good idea. I work in a pnpm shop, and not being able to use that for install during push is not only putting us as a speed advantage, but it's forcing us to use a codepath that we didn't choose.

Describe the solution you'd like

I'd like first-class support for modifying the install command that's used, or being able to skip the install step altogether, allowing the app's amplify:{appName} script to prepare the directory for push. e.g. copying/moving node_modules into the target directory, or running our installer of choice.

Describe alternatives you've considered

There aren't any. Woo!

Additional context

n/a

[kaustavghosh06]

We recently added support for installation of CLI as an executable using curl - curl -sL https://aws-amplify.github.io/amplify-cli/install | bash && $SHELL which doesn't require installation of npm or yarn. You can learn about more installation options here - https://docs.amplify.aws/cli/start/install

[shellscape]

Thanks for the reply, but that's not quite what I was talking about. Amplify will autodetect package-lock.json or yarn.lock and install accordingly. I was speaking towards that process. Related: #6035 (comment)

[mattfysh]

I'm running into some issues now too, due to the use of workspace: in package.json

When I run amplify push it tries to do an npm install in the project root, which fails.

For now, I've created an executable script and run amplify push as follows:

PATH=$PWD/scripts:$PATH amplify push --yes

And then in scripts/npm I've got:

#!/bin/bash

set -e

pnpm "$@"

[vjpr]

NOTE: If you want to install this cli locally (and you are not using hoist=true) you need to add this to your pnpmfile.

.pnpmfile.cjs#readPackage

  if (pkg.name === '@aws-amplify/cli') {
    pkg.dependencies = {
      ...pkg.dependencies,
      'uuid': '^3.2.1', // Must be 3, not 4.

      // Dynamic requires.
      // See `/node_modules/.pnpm/@[email protected]/node_modules/@aws-amplify/cli/lib/utils/post-install-initialization.js.`
      'amplify-dynamodb-simulator': '*',
      'amplify-frontend-ios': '*',
      'amplify-go-function-runtime-provider': '*',
      'amplify-java-function-runtime-provider': '*',
      'amplify-python-function-runtime-provider': '*',
    }
  }

    ////////////////////

    // When running `aws configure` this list is generated.
    const officialPlugins = [
      'amplify-provider-awscloudformation',
      'amplify-category-analytics',
      'amplify-category-api',
      'amplify-category-auth',
      'amplify-category-function',
      'amplify-category-hosting',
      'amplify-console-hosting',
      'amplify-container-hosting',
      'amplify-category-interactions',
      'amplify-category-notifications',
      'amplify-category-predictions',
      'amplify-category-storage',
      'amplify-category-xr',
      'amplify-codegen',
      'amplify-frontend-flutter',
      'amplify-frontend-android',
      'amplify-frontend-ios',
      'amplify-frontend-javascript',
      'amplify-util-mock',
    ]

    /*
    "peerDependenciesMeta": {
      "node-sass": {
        "optional": true
      }
    },
    */
    pkg.peerDependenciesMeta = officialPlugins.reduce((acc, pkg) => {
      acc[pkg] = {optional: true}
      return acc
    }, {})

    ////////////////////

  if (pkg.name === 'amplify-category-auth') {
    pkg.dependencies = {
      ...pkg.dependencies,
      'cfn-response': '*',
      'express': '*',
      'body-parser': '*',
      'aws-serverless-express': '*',
      '@aws-cdk/aws-iam': '*',
      '@aws-cdk/aws-lambda': '*',
      '@aws-cdk/core': '*',
      'axios': '*',
    }
  }

Amplify should explicitly declare their dependencies in package.json. They also use a bunch of dynamic requires.

All the plugins should be declared as optional peer dependencies using peerDependenciesMeta.

[armenr]

+1 to what @vjpr has mentioned.

NOTE: If you want to install this cli locally (and you are not using hoist=true) you need to add this to your pnpmfile.

.pnpmfile.cjs#readPackage

  if (pkg.name === '@aws-amplify/cli') {
    pkg.dependencies = {
      ...pkg.dependencies,
      'uuid': '^3.2.1', // Must be 3, not 4.

      // Dynamic requires.
      // See `/node_modules/.pnpm/@[email protected]/node_modules/@aws-amplify/cli/lib/utils/post-install-initialization.js.`
      'amplify-dynamodb-simulator': '*',
      'amplify-frontend-ios': '*',
      'amplify-go-function-runtime-provider': '*',
      'amplify-java-function-runtime-provider': '*',
      'amplify-python-function-runtime-provider': '*',
    }
  }

    ////////////////////

    // When running `aws configure` this list is generated.
    const officialPlugins = [
      'amplify-provider-awscloudformation',
      'amplify-category-analytics',
      'amplify-category-api',
      'amplify-category-auth',
      'amplify-category-function',
      'amplify-category-hosting',
      'amplify-console-hosting',
      'amplify-container-hosting',
      'amplify-category-interactions',
      'amplify-category-notifications',
      'amplify-category-predictions',
      'amplify-category-storage',
      'amplify-category-xr',
      'amplify-codegen',
      'amplify-frontend-flutter',
      'amplify-frontend-android',
      'amplify-frontend-ios',
      'amplify-frontend-javascript',
      'amplify-util-mock',
    ]

    /*
    "peerDependenciesMeta": {
      "node-sass": {
        "optional": true
      }
    },
    */
    pkg.peerDependenciesMeta = officialPlugins.reduce((acc, pkg) => {
      acc[pkg] = {optional: true}
      return acc
    }, {})

    ////////////////////

  if (pkg.name === 'amplify-category-auth') {
    pkg.dependencies = {
      ...pkg.dependencies,
      'cfn-response': '*',
      'express': '*',
      'body-parser': '*',
      'aws-serverless-express': '*',
      '@aws-cdk/aws-iam': '*',
      '@aws-cdk/aws-lambda': '*',
      '@aws-cdk/core': '*',
      'axios': '*',
    }
  }

Amplify should explicitly declare their dependencies in package.json. They also use a bunch of dynamic requires.

All the plugins should be declared as optional peer dependencies using peerDependenciesMeta.

[mikhael28]

I'm going to say +1 to asking for first class support for pnpm. Meaning, in the Build Settings of the Amplify Console, support having pnpm install instead of npm or yarn. Frankly, this shouldn't take too much work - whatever your base image is that runs all of these builds, just install pnpm as one of the initial dependencies. My open-source project just moved to pnpm (loving it, saving space/data from redownloading node_modules) and we are trying to build our app to keep our deployment pipeline going. We also changed to Vite at the same time, so that could be inviting some trouble as well.

Either way, we are going to be looking at migrating to another system ASAP for our CI pipeline - trying to use npm or yarn install in the Build settings successfully generates a build... but the hilarious thing is that the build is actually broken, even though Amplify puts it through. That's a lack of testing on our part, but still - hilarious.

[jayeclark]

Oh hey @mikhael28 fancy seeing you here. 😆

First class support would be great, but (commenting here so it can help others if applicable) a workaround that seems to be fairly effective (it solved our deploy issues) is to edit the amplify.yml build file to use npx pnpm install for preBuild and npx pnpm build for build, instead of their yarn or npm counterparts.

[mikhael28]

@jayeclark is a boss 👩‍💻

[hkbertoson]

@jayeclark Thanks! This worked great.

[tatethurston]

pnpm is supported by Node.js's Corepack so it's reasonable for customers to expect first class pnpm support, alongside yarn and npm.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[tatethurston]

I see this was closed, but using pnpm appears to still not have first class support like npm and yarn

pnpm: command not found

[stefanofa]

@siegerts since as I understand you are working on a new default image for amplify, perhaps it might be a good time to consider adding pnpm as well!

[kearfy]

Heya! Would just like to point out that you can currently use pnpm by including npm i -g pnpm as the first step in the preBuild steps. Did not see this mentioned in the thread so I though I'd mention it!

[tatethurston]

Heya! Would just like to point out that you can currently use pnpm by including npm i -g pnpm as the first step in the preBuild steps. Did not see this mentioned in the thread so I though I'd mention it!

Yes but that’s slower because you have to download it each time, and surprising given yarn and npm have first class support.

[kearfy]

I am well aware, but I still rather do that then use npm/yarn 😁

[higherstates]

Here's my current approach to PNPM Not found error while building:

version: 1
frontend:
  phases:
    # IMPORTANT - Please verify your build commands
    preBuild:
      commands:
        - npm install -g pnpm # This line installs pnpm globally
        - pnpm install # Install project dependencies
    build:
      commands: 
        - pnpm run build
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: dist
    files:
      - '**/*'
  cache:
    paths: []

[Benjamin-Dobell]

preBuild is currently the way to go, but I'd suggest using corepack and the packageManager field for consistency. Basically you pin the PNPM version to what you're using in development, which is handy for collaborators working on your project too.

i.e.

package.json

{
  ...
  "engines": {
    "node": ">=20"
  },
  "packageManager": "[email protected]"
}

amplify.yml

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - corepack enable
        - pnpm install
   # ...

Note: You can grab the implicitly generated amplify.yml as a starting point from the "Build Settings" tab in the Amplify web console.

[renchris]

Is there a way to fork the Amplify image, install pnpm, and use that, so we have pnpm installed once in the image and not re-installed every build?

[renchris]

I created a custom docker image renchris/amazonlinux:pnpm with the base Amazon Linux 2023 image and pnpm dependency: https://hub.docker.com/repository/docker/renchris/amazonlinux/general

You can use it by going to Build Settings → Edit → Click Build Image dropdown Amazon Linux 2023(default) then select Custom Build Image → type in renchris/amazonlinux:pnpm as the image → click Save

I created the Docker Image by:

• opening and connecting to a new EC2 Amazon Linux instance
• install docker on the EC2 instance with commands:

sudo yum install docker -y
sudo systemctl start docker
sudo systemctl enable docker
docker --version
sudo usermod -a -G docker $(whoami)
newgrp docker

• pull and run the Amazon Linux 2023 image with commands:

docker pull amazonlinux
docker run -it amazonlinux:latest

• install pnpm and the custom build image requirements dependencies with commands:

yum update
yum install -y git openssh bash wget tar
curl -sL https://rpm.nodesource.com/setup_18.x | bash -
yum install -y nodejs
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
source ~/.nvm/nvm.sh
curl -fsSL https://get.pnpm.io/install.sh | sh -
source /root/.bashrc
yum clean all
exit

• commit and upload the image to Docker Hub with commands:

docker ps -a
docker commit 7c4fa070f776 amazonlinux-pnpm
docker images
docker tag amazonlinux-pnpm renchris/amazonlinux:pnpm
docker login
docker push renchris/amazonlinux:pnpm

Replace the commit ID (ie. 7c4fa070f776), local published image name (ie. amazonlinux-pnpm), and Docker Hub image name (ie. renchris/amazonlinux:pnpm), with your own

Now having this Custom Build Image, we have pnpm installed and don't to run npm install -g pnpm for every build. This reduced my app build time by 65 seconds!

[neil-rubens]

seems that there is an official guide on how to use pnpm for amplify (although for mono repo):
https://docs.aws.amazon.com/amplify/latest/userguide/monorepo-configuration.html#turborepo-pnpm-monorepo-configuration