Support ES import when amplify mock function

[bc4253]

Is this feature request related to a new or existing Amplify category?

function

Is this related to another service?

No response

Describe the feature you'd like to request

Similar to issue: #5691 support for import when mocking a function locally.

Describe the solution you'd like

Either support ES import by default or support adding "type": "module" to package.json for amplify functions when testing with amplify mock function

Describe alternatives you've considered

currently the only alternative is to require modules

Additional context

No response

Is this something that you'd be interested in working on?

• 👋 I may be able to implement this feature request

Would this feature include a breaking change?

• ⚠️ This feature might incur a breaking change

[josefaidt]

Hey @bc4253 👋 thanks for taking the time to file this! I've marked this as a feature request for the team to evaluate further 🙂

[amit-ecovoice]

this feature request invalidate the steps mentioned in - https://docs.amplify.aws/cli/function/#graphql-from-lambda

these steps won't work without this feature. so link needs to be updated.

[amit-ecovoice]

I'm trying to follow steps mentioned here - https://docs.amplify.aws/cli/function/#graphql-from-lambda to call GraphQL API from lambda.
As soon as I add "type": "module" to package.json , it starts throwing an error on amplify mock command. As per above link I need to import node-fetch but I'm not able to do it. Lambda code is just a hello-world code as of now, just changing package.json starts creating this problem.

stack: 'Error: Could not load lambda handler function due to Error [ERR_REQUIRE_ESM]: Must use import to load ES Module

[alexburciu]

The docs should also show how to run the function to try the GraphQL API (with and without amplify mock).

[josefaidt]

Hey folks 👋 here is a CommonJS version of the function sample

const crypto = require('@aws-crypto/sha256-js');
const { defaultProvider } = require('@aws-sdk/credential-provider-node');
const { SignatureV4 } = require('@aws-sdk/signature-v4');
const { HttpRequest } = require('@aws-sdk/protocol-http');
const fetch = require('node-fetch');

const { Request } = fetch
const { Sha256 } = crypto;
const GRAPHQL_ENDPOINT = process.env.API_<YOUR_API_NAME>_GRAPHQLAPIENDPOINTOUTPUT;
const AWS_REGION = process.env.AWS_REGION || 'us-east-1';

const query = /* GraphQL */ `
  query LIST_TODOS {
    listTodos {
      items {
        id
        name
        description
      }
    }
  }
`;

/**
 * @type {import('@types/aws-lambda').APIGatewayProxyHandler}
 */
exports.handler = async (event) => {
  console.log(`EVENT: ${JSON.stringify(event)}`);

  const endpoint = new URL(GRAPHQL_ENDPOINT);

  const signer = new SignatureV4({
    credentials: defaultProvider(),
    region: AWS_REGION,
    service: 'appsync',
    sha256: Sha256
  });

  const requestToBeSigned = new HttpRequest({
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      host: endpoint.host
    },
    hostname: endpoint.host,
    body: JSON.stringify({ query }),
    path: endpoint.pathname
  });

  const signed = await signer.sign(requestToBeSigned);
  const request = new Request(endpoint, signed);

  let statusCode = 200;
  let body;
  let response;

  try {
    response = await fetch(request);
    body = await response.json();
    if (body.errors) statusCode = 400;
  } catch (error) {
    statusCode = 500;
    body = {
      errors: [
        {
          message: error.message
        }
      ]
    };
  }

  return {
    statusCode,
    headers: {
          "Access-Control-Allow-Headers" : "Content-Type",
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "OPTIONS,POST,GET"
        },
    body: JSON.stringify(body)
  };
};

Two notes here:

1. we need to install node-fetch v2 rather than v3. Version 3 is ESM-only
2. we need to remove "type": "module" or change it to "type": "commonjs"

For testing with mock we can use a dotenv file to manually provide/override environment variables to the function https://docs.amplify.aws/cli/usage/mock/#function-mock-environment-variables

[ghost]

Using commonjs was causing headaches for me.

I have come up with a workaround using Typescript. I am posting my retrospective notes here in hopes it helps others get around this issue if they are willing to use Typescript to solve it or are already using Typescript.

Expand this for Typescript setup if you don't already it set up for your function(s)

make sure tsc is installed globally

npm install -g typescript

in amplify project root package.json I added to "scripts"

    "amplify:myFunctionName": "cd amplify/backend/function/myFunctionName/src && tsc"

i created this tsconfig.json inside of amplify/backend/function/myFunctionName/src

{
    "compilerOptions": {
      "target": "es2017",
      "module": "commonjs",

      /*  allows us to use ESM */
      "esModuleInterop": true,
      /* allow use of import AWS from 'aws-sdk' instead of import * as AWS from 'aws-sdk' */
      "allowSyntheticDefaultImports": true,
    },
    "include": ["."],
    "exclude": ["node_modules", "**/*.spec.ts"]
  }

The esModuleInterop: true and allowSyntheticDefaultImports: true allow me to use ES Module pattern in typescript while the output uses CommonJS that is supported by the errant node14 that Mock seems to use (see #10940)

If you have typescript setup already, you can seemingly just add these 2 lines to "compilerOptions" in tsconfig.json.

      /* allows us to use ESM */
      "esModuleInterop": true,
      /* allow use of import AWS from 'aws-sdk' instead of import * as AWS from 'aws-sdk' */
      "allowSyntheticDefaultImports": true,

now in index.tsx I can use normal import statements

import AWS from "aws-sdk";
import { v4 as uuid4 } from "uuid";

and in other ts files in that same directory I can use normal module pattern exports

export const foo = () => {}

Overall this has made a huge difference for me to be able to use ESM using amplify mock myFunctionName

It would be nice if we could use the cli to create new amplify functions using a typescript template so I don't have to do as much manual package.json updates and copy-pasting the tsconfig from function to function.

[alex-griffiths]

I'm running into some similar issues trying to mock lambda functions using TypeScript. As a minimal example, I can create a function using amplify function add called testfunc. Then changing the index.js to an index.mts to get my TypeScript file.

I add 'type': 'module' to my package.json and use the following tsconfig.json

{
    "compilerOptions": {
        "target": "ES2020", /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
        "rootDir": "./", /* Specify the root folder within your source files. */
        "moduleResolution": "node", /* Specify how TypeScript looks up a file from a given module specifier. */
        "sourceMap": false, /* Create source map files for emitted JavaScript files. */
        "noEmit": false, /* Disable emitting files from a compilation. */
        "preserveConstEnums": true, /* Disable erasing 'const enum' declarations in generated code. */
        "isolatedModules": true, /* Ensure that each file can be safely transpiled without relying on other imports. */
        "allowSyntheticDefaultImports": true, /* Allow 'import x from y' when a module doesn't have a default export. */
        "esModuleInterop": true, /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables 'allowSyntheticDefaultImports' for type compatibility. */
        "forceConsistentCasingInFileNames": true, /* Ensure that casing is correct in imports. */
        "strict": true, /* Enable all strict type-checking options. */
        "skipLibCheck": true /* Skip type checking all .d.ts files. */
    },
}

(I have used a tsconfig.json similar to the one @ChrisColemanEH suggested, however without esModuleInterop and allowSyntheticDefaultImports and had the same result)

Compiling this with tsc gives me my index.mjs file as expected, which looks like:

/**
 * @type {import('@types/aws-lambda').APIGatewayProxyHandler}
 */
export const handler = async (event) => {
    console.log(`EVENT: ${JSON.stringify(event)}`);
    return {
        statusCode: 200,
        //  Uncomment below to enable CORS requests
        //  headers: {
        //      "Access-Control-Allow-Origin": "*",
        //      "Access-Control-Allow-Headers": "*"
        //  }, 
        body: JSON.stringify('Hello from Lambda!'),
    };
};

When I run amplify mock function testfunc I get the following output:

Starting execution...
testfunc failed with the following error:
{
  stack: "Error: Could not load lambda handler function due to Error: Cannot find module 'C:\\path\\to\\code\\amplify\\backend\\function\\testfunc\\src\\index'\n" +
    'Require stack:\n' +
    '- C:\\snapshot\\repo\\build\\node_modules\\amplify-nodejs-function-runtime-provider\\lib\\utils\\execute.js\n' +
    '    at loadHandler (C:\\snapshot\\repo\\build\\node_modules\\amplify-nodejs-function-runtime-provider\\lib\\utils\\execute.js:106:15)\n' +
    '    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n' +
    '    at async invokeFunction (C:\\snapshot\\repo\\build\\node_modules\\amplify-nodejs-function-runtime-provider\\lib\\utils\\execute.js:61:27)\n' +
    '    at async process.<anonymous> (C:\\snapshot\\repo\\build\\node_modules\\amplify-nodejs-function-runtime-provider\\lib\\utils\\execute.js:32:24)',
  message: "Could not load lambda handler function due to Error: Cannot find module 'C:\\path\\to\\code\\amplify\\backend\\function\\testfunc\\src\\index'\n" +
    'Require stack:\n' +
    '- C:\\snapshot\\repo\\build\\node_modules\\amplify-nodejs-function-runtime-provider\\lib\\utils\\execute.js'
}
Finished execution.

Amplify CLI version: 10.5.0
Local NodeJS version: 16.15.0

// package.json
{
  "name": "testfunc",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.mjs",
  "license": "Apache-2.0",
  "type": "module",
  "devDependencies": {
    "@types/aws-lambda": "^8.10.92"
  }
}

If I create this function and don't change anything about the generated files (i.e. not converting to TypeScript), mocking the function works fine and I see the response and console output as expected. I also know from experience with some other functions in another project, that running code similar to the example provided by @josefaidt by using ESM instead of CJS will run correctly on AWS, though I have not tried mocking those functions.

[AsemK]

+1
I hope this gets fixed soon.

[erikash]

Maybe it could help others facing the same issue:
Personally I just stopped using amplify mock function and just use ts-node to run my function directly and it just works :)

[joyacv2]

Hi, needs to be fixed, please

[unknown1337]

would be really helpfull...

[wheresthecode]

I think this is a bug and not a feature request. If you use the "amplify add function" along with the graphQL template it will generate a .js file that uses ES imports. That stock template does not work for amplify mock.

[scherrer]

Agreed, this should be classified as a bug. The template/boilerplate examples cannot be run using the mock functionality.

[johnemcbride]

plz fix

[raphipaffi]

Suffering from the same bug. Currently, I need to push any change to AWS to test it. Here is the output from amplify mock function:

geocodeandaddclient failed with the following error:
{
  stack: 'Error: Could not load lambda handler function due to Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/index.js\n' +
    'require() of ES modules is not supported.\n' +
    'require() of /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/index.js from /snapshot/repo/build/node_modules/amplify-nodejs-function-runtime-provider/lib/utils/execute.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.\n' +
    'Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/package.json.\n' +
    '\n' +
    '    at loadHandler (/snapshot/repo/build/node_modules/amplify-nodejs-function-runtime-provider/lib/utils/execute.js:106:15)\n' +
    '    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n' +
    '    at async invokeFunction (/snapshot/repo/build/node_modules/amplify-nodejs-function-runtime-provider/lib/utils/execute.js:61:27)\n' +
    '    at async process.<anonymous> (/snapshot/repo/build/node_modules/amplify-nodejs-function-runtime-provider/lib/utils/execute.js:32:24)',
  message: 'Could not load lambda handler function due to Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/index.js\n' +
    'require() of ES modules is not supported.\n' +
    'require() of /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/index.js from /snapshot/repo/build/node_modules/amplify-nodejs-function-runtime-provider/lib/utils/execute.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.\n' +
    'Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/raphael/Projects/route-planner/amplify/backend/function/geocodeandaddclient/src/package.json.\n'
}
Finished execution.

My package.json

{
  "name": "geocodeandaddclient",
  "type": "module",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "@mapbox/mapbox-sdk": "^0.15.0"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.92"
  },
  "engines": {
    "node": ">=18.0.0"
  }
}

using a Mac M1 with
amplify 10.6.2
node 18.14.1

[hackmajoris]

@josefaidt I think that the support of AWS JavaScript SDK v3 should be prioritised for Amplify CLI. Right now is really big mess, between: TypeScript configs, CommonsJS and Modules node mode.

Running locally the Amplify mock function for example, gives this info message:

(node:8692) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).

I did try to configure SDK for JavaScript(v3) within Amplify context with no success.

[thekamahele]

nearly one year later...please fix.

[scherrer]

please fix this bug. It slows development so much.

[johnemcbride]

wen fix?

[fvisticot]

any update ? would be really useful to speed up devs !

[skjortan23]

Yes just starting a new project and getting the exact same error. Having working local mock functionality for generated code should be a mimimum.

Might have to take the advice in the thread and do a .ts config etc.

But this is a functionality bug as i see it as well.

[skjortan23]

For people having the same issue here. Mocking of typescript functions does not seem to work. However they seem to run with amplify push.

So the workaround is as mentions above by @erikash is to just test locally with ts-node. and to create a wrapper that calls your function for test. given the event

[mah77ss]

+1
I also hope the issue gets resolved.

[peter-iglaev]

any updates?

AWS Developers, you must to respect your users. say smth!

[vorant94]

any updates?

AWS Developers, you must to respect your users. say smth!

nope, they would better rewrite the whole project structure and call it "Next gen dev experience" rather than fix things that developers face once they make one step out from the way the product was advertised

[hackmajoris]

They always seems to put marketing over everything else.

[adrianpunkt]

Sadly, amplify is good only for todo tasks kind of projects, anything more serious than that is a waste of time and money. Been there, done that, I'd recommend staying away from it under any circumstance if your intention is to use it for business.

[cjsilva-umich]

Could anyone from the aws team comment on this? How can you all let issues like this go for so long?

[wac-eric]

Still waiting on this bug to be resolved as well. For now, I'm testing by pushing it to cloud and checking it on the lambda online...

[hackmajoris]

Just to note: using Lambda Layers for Amplify functions will also break the mock functionality. It's clear that this isn't solely an Amplify problem; it's a generic one. But, I don't see how to avoid using Lambda Layers for a larger project. Consequently, mocking becomes obsolete, which presents a significant problem.

For the Amplify Team: It would be really helpful to add some documentation and at least mention this limitation. Additionally, having documentation related to testing and debugging Amplify functions would be very beneficial.

[vorant94]

Just to note: using Lambda Layers for Amplify functions will also break the mock functionality.

If you use TypeScript you can workaround sharing code without Lambda Layers with the help of multiple root dirs inside your tsconfig.json

here is the post about it https://www.vorant94.io/posts/aws-amplify-functions-on-steroids#sharing-code-between-lambdas

and here is the repo with example https://github.com/vorant94/amplify-functions-on-steroids

[hackmajoris]

Just to note: using Lambda Layers for Amplify functions will also break the mock functionality.

If you use TypeScript you can workaround sharing code without Lambda Layers with the help of multiple root dirs inside your tsconfig.json

here is the post about it https://www.vorant94.io/posts/aws-amplify-functions-on-steroids#sharing-code-between-lambdas

and here is the repo with example https://github.com/vorant94/amplify-functions-on-steroids

I use Nx libs for code sharing, but it's not what Lambda Layers solves. The problem that Lambda Layers solves is deploying all the dependencies once, so that it doesn't require a separate build or dependency installation for each Lambda.

I also remember that Amplify has a size limit per Function, and deploying everything without Lambda dependencies was a problem.

[vorant94]

The problem that Lambda Layers solves is deploying all the dependencies once, so that it doesn't require a separate build or dependency installation for each Lambda.

yeah, that’s why u called it workaround)

I also remember that Amplify has a size limit per Function, and deploying everything without Lambda dependencies was a problem.

Interesting… I remember this limit to be on Lambda side, not Amplify. And this limit is actually pretty high, I played with it and put the whole express service with set of basic libraries there without reaching even half of it…

[hackmajoris]

The problem that Lambda Layers solves is deploying all the dependencies once, so that it doesn't require a separate build or dependency installation for each Lambda.

yeah, that’s why u called it workaround)

I also remember that Amplify has a size limit per Function, and deploying everything without Lambda dependencies was a problem.

Interesting… I remember this limit to be on Lambda side, not Amplify. And this limit is actually pretty high, I played with it and put the whole express service with set of basic libraries there without reaching even half of it…

The maximum size for a .zip deployment package for Lambda is 250 MB.

[vorant94]

The maximum size for a .zip deployment package for Lambda is 250 MB.

yeah, and if during build process exclude dev dependencies it is more than enough to include fully-fledged express server all more so for some shared typescript code

[SilverLinings89]

It is important to note that the aws-sdk version 2 deprecation has been announced. The recommended fix is to migrate to v3. v3 uses import. import doesn't work with amplify function mock. Can somebody please clarify how one is supposed to use this now?

[josefaidt]

Hey folks just wanted to stop by and mention that we have not forgotten about this issue. As others in the thread have mentioned you can use a small harness like an express/hono/polka server to replace mock given the limitation described in this issue. To share and bundle dependencies across Functions you can use a tool like esbuild to bundle and output to the Function's src/ directory.

@SilverLinings89 for SDK v3 they support CommonJS and ECMAScript Modules -- for the most part you can swap the import...from for const...require
https://github.com/aws/aws-sdk-js-v3/tree/main/clients/client-cognito-identity-provider#import

[SilverLinings89]

@josefaidt Thanks for the clarification, we have the code working that way in our codebase already. My point was more along the lines that the examples/code excerpts in the AWS SDK docs will no longer be compatible/useable in functions that need to use amplify mock (out of the box). There's afaik no clear indication in the docks that using an import will break mock. Considering this chapter, for example: https://docs.amplify.aws/javascript/tools/cli/usage/mock/#function-mock-limitations there are mentions of limitations but this pretty big limitation isn't referenced but might be relevant for newcomers.