22

Recently I found that aws-sdk NPM module is preinstalled in AWS Lambda nodejs8.10. And I can't find any information in the internet about it.

Which other node.js modules are pre-installed in AWS Lambda?

1
  • 2
    docs.aws.amazon.com/lambda/latest/dg/… says: "Note The Lambda service has preinstalled the AWS SDK for Node.js." I think its safe to assume that that is the only thing pre-installed. Commented Nov 30, 2018 at 23:57

5 Answers 5

20

Only the aws-sdk package is preinstalled .

All the rest is loaded from the "node_modules" directory..

You can find information about it here:

https://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html

Update

Now that the aws-sdk package is deprecated, newer runtime versions (since nodejs18.x) only have @aws-sdk/* packages installed, instead of aws-sdk.

Sign up to request clarification or add additional context in comments.

3 Comments

I inadvertently loaded up a Lambda function the other day in which I forgot to include aws-sdk in my node_modules. Was quite surprised it worked and upon searching found this post. As I update various skills I am going to pull my own copy of aws-sdk from deployment. Considering the size of aws-sdk and dependencies I am glad AWS is preinstalling it. I would be curious to know when/if they update it or if it is frozen at the version that was released when you deploy.
Is the aws-sdk version listed anywhere? Or we can assume it always uses the latest?
As noted below, this answer doesn't quite tell the whole story. obviously the dependencies of aws-sdk are preinstalled as well, such as lodash (though you shouldn't rely on that)
14

Unfortunately the docs don't seem very specific about it, but you can grab their docker images yourself and bash in to inspect them:

docker run --rm -it --entrypoint /bin/bash amazon/aws-lambda-nodejs:14

The Dockerfile has ENTRYPOINT ["/lambda-entrypoint.sh"] and by inspecting that file I was able to determine that it runs Node via /var/runtime/bootstrap.

/var/runtime/bootstrap adds various directories to NODE_PATH from which modules can be loaded:

if [ -z "$NODE_PATH" ];
then
  nodejs_mods="/opt/nodejs/node_modules"
  nodejs14_mods="/opt/nodejs/node14/node_modules"
  runtime_mods="/var/runtime/node_modules"
  task="/var/runtime:/var/task"
  export NODE_PATH="$nodejs14_mods:$nodejs_mods:$runtime_mods:$task"
fi

However /opt/nodejs doesn't exist in the image and /var/task is where Lambda puts your custom code.

So /var/runtime/node_modules is the only directory we need to inspect:

bash-4.2# cd /var/runtime
bash-4.2# npm ls
/var/runtime
└─┬ [email protected]
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  ├── [email protected] extraneous
  └── [email protected] extraneous

As you can see, only aws-sdk is preinstalled. But you could use this process to determine if they add more preinstalled deps in future versions of their Node.js runtime.

UPDATE

I repeated this process on amazon/aws-lambda-nodejs:20 and confirmed that now the AWS Node SDK v3 packages are installed instead:

bash-5.2# npm list
npm notice 
npm notice New minor version of npm available! 10.5.2 -> 10.8.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.1
npm notice Run npm install -g [email protected] to update!
npm notice 
/var/runtime
├── @aws-sdk@ extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous
├── @aws-sdk/[email protected] extraneous

Comments

6

I couldn't find an official list so I wrote a script to create a list.

I have a number of other libraries installed in my Lambda, those are (excluding built-in nodejs modules which are also available of course):

'aws-sdk',      <-- preinstalled by AWS
'awslambda',
'base64-js',
'dynamodb-doc',
'ieee754',
'imagemagick',
'isarray',
'jmespath',
'lodash',
'sax',
'uuid',
'xml2js',
'xmlbuilder'

Code to generate this list:

function flatten(arrayOfArrays) {
    return Array.prototype.concat.apply([], arrayOfArrays)
}

function onlyUnique(value, index, self) {
    return self.indexOf(value) === index;
}

function extPackageNames(node) {
    if (!node.children) return [];
    const arrayOfArrays = node.children.map(c => [c.name].concat(extPackageNames(c)))
    const result = flatten(arrayOfArrays)
    return result
}

exports.handler = async (event) => {
    const rpt = require("read-package-tree")
    const module = require("module")

    const pathArg = process.env.NODE_PATH
    const allPaths = pathArg.split(":")

    // '/var/task' is this package on lambda
    const externalPaths = allPaths.filter(p => p !== "/var/task")

    // read all package data
    const ps = externalPaths.map((path) => rpt(path).catch(err => err))
    const rpts = await Promise.all(ps).catch(err => err)

    // extract the information we need
    const packagesPerPath = rpts.map(extPackageNames)
    const countPerPath = packagesPerPath.map(arr => arr.length)
    const packages = flatten(packagesPerPath)

    // remove duplicates
    const uniquePackages = packages.filter(onlyUnique)

    // remove node.js built-in modules
    const uniqueCustomPackages = uniquePackages.filter(p => !module.builtinModules.includes(p))

    const result = {
        node_path: pathArg,
        paths: externalPaths.map((e, i) => [e, countPerPath[i]]),
        uniqueCustomPackages
    }

    console.log(result)

    const response = {
        statusCode: 200,
        body: JSON.stringify(result)
    };
    return response;
};

To run this on lambda you will need to zip it together with a node_modules folder containing read-package-tree.

6 Comments

Is this not useful, or missing something?
You've listed the sub-dependencies of aws-sdk. github.com/aws/aws-sdk-js/blob/… Relying on or require-ing those is a dangerous because those are internal to aws-sdk library. Also, the imagemagick in your list looks suspicious, it's a fairly large library. It might be coming from your Lambda Layers - docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html which is out of the scope of this question.
Thanks for the response. Well, that's the total list of installed modules, so of course it would contain the aws-sdk dependencies. I don't have any layers configured. I think image processing is a common use case, that's why imagemagick is preinstalled.
imagemagick was not preinstalled for me. I created a "from scratch" Lambda function with the Node.js 12.x. runtime. Adding im = require('imagemagick'); caused the function to fail with Error: Cannot find module 'imagemagick'.
note that at the time the above list was generated it was probably still run under node 10.x. . the list should not be taken as a guarantee to exist anyway, as it could change at any time. This answer was more about an idea for a script to find out what is installed.
|
3

I've used the "https" and the "url" package, so those at least is pre-installed. There are quite a few standard node.js modules which need a native layer.

Clearly the AWS modules are in there, for communicating with AWS services. I've used SQS, for example.

Haven't tried "fs" yet, but since it requires a native layer, and is something you might want to do (e.g. persisting stuff to /tmp) I'm assuming it's there.

Somewhere there ought to be a list. But I can't find one. Guess you just have to try, and if the requires fails, then you need to put a module in node_modules, then see if it demands dependencies.

2 Comments

Here is a list of node.js built in modules github.com/sindresorhus/builtin-modules/blob/master/…
Thanks. That's really useful. Looks like the Lambda framework adds at least a few more, like the aws library.
0
  • For Lambda functions still running with NodeJS 16. AWS SDK v2 is pre-included.
  • For Lambda functions running from NodeJS 18 and 20, AWS SDK V3 is pre-included.

What this means is that for tenants who:

  1. Haven't updated their codebase to use SDK v3 and
  2. Need to transition to NodeJS 18 or 20 from NodeJS 16 and below
  3. Didn't updated their package.json files to include SDK v2 explicitly .

will get an error related to missing node_modules post transition.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.