Lambda total size limitation

[sashker]

Thanks for your PoC, guys. Though your article has minor differences from the git repo, I managed to run it.

Did you hit the problem with the total size of a lambda 250Mb including all layers? If we build all handlers in one repo this might become a problem very quickly with modules size.

[Envek]

No, we are quite far from hitting this limit. But we're paying extra attention for our node_modules size and choosing packages with account for their size and number of dependencies.

As for now layer with runtime dependencies weights about 70 MiB unarchived. And we have quite a lot of packages there: graphql client, http client, openapi validator, html parser, multipart request parser, …

Every function itself weights just about dozens or hundreds of kilobytes because it contains only handler for this function and files that this particular handler uses. See Bonus: Put only relevant code into your Lambdas functions part of our blog post.

You can check your layer and function weights with command like this:

sam build
du -h -d 1 .aws-sam/build/ | sort -h -r

Also note that these numbers doesn't just sum up as deployment package for every lambda consists from only this Lambda folder and layers that it uses. So yes, you can keep all your handlers in one repo.

[sashker]

Actually, my dependency list is quite minimalistic.

"dependencies": {
        "@types/node-fetch": "^2.5.10",
        "node-fetch": "^2.6.1",
        "source-map-support": "^0.5.19"
    },
    "devDependencies": {
        "@tsconfig/node14": "^1.0.0",
        "@types/aws-lambda": "^8.10.72",
        "@types/node": "^14.14.26",
        "jest": "^26.6.3",
        "typescript": "^4.1.5"
    },

And in this configuration, local node_modules folder is around 115Mb, and strangely enough

2,8M	.aws-sam/build/RuntimeDependenciesLayer
2,8M	.aws-sam/build

But when I download a zip-archive created by sam package it's also 115+ Mb.

I understand how it works in general, but moving from the Golang environment doesn't go as smooth as hoped, so maybe I'm missing something ))

P.S. It seems we don't need the typescript and all the bells and whistles related to it within the DependencyLayer if we transpile the code locally and just run it, which gives extra 70-80Mb

[Envek]

Hmm. Just rechecked what is uploaded in SAM CLI's S3 bucket on sam package (and what was uploaded on the latest deploy) and found one 10 MB zipped archive for dependencies layer (around 70 MiB) and zip archive for every handler (dozens of kilobytes each). You can find your S3 bucket in samconfig.toml

Most probably you have something misconfigured. Run sam build and examine contents of .aws-sam/build/ directory. That's what is uploaded to S3 after zipping, actually.

---

devDependencies doesn't go into runtime dependencies layer if you use yarn install --production or npm install --production to build it.

And you can move all @types/* packages to devDependencies as they're needed only at build time (unless you're using some typescript enums in runtime, but it is rare).

[sashker]

Cheers, Andrey.
I actually found my fault - I was using sam package command from the Golang pipeline that obviously included my local bloated node_mudules in a development configuration :)

So, .aws-build size at the moment, less than 10Mb.

Thanks a lot!

[Envek]

However, if your dependencies (or whatever) is really huge, you can switch to Lambda Container Images. Size limit for those is 10 GB.

AWS SAM CLI supports them starting from version 1.13.1). And here is the manual: Using container image support for AWS Lambda with AWS SAM

But we don't have experience with Lambda container images yet.