Use sharp on lambda@edge

Question

I'm trying to use sharp on a couple of AWS lambda@edge. The idea is to resize and cache an image when requested (see this).

I'm also using serverless with serverless-webpack to deploy the lambdas.

I can deploy the lambdas and everything goes well if I test them in AWS console.

However, these are lamda@edge and they will be used as cloudwatch request/response triggers. Therefore, the maximum lambda size is 1Mb.

My problem is I can't seem to get even near that size, the best I could achieve was 11.6Mb. And, it seems it's possible as seen in that first link.

This is the serverless configuration which results in 34.7Mb lambda:

custom:
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
    packagerOptions:
      scripts:
        - rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp

package: 
  exclude:
    - .env
    - .git/**
    - .gitlab-ci.yml
    - tests*
  excludeDevDependencies: true
  individually: true

And with this I got 11.6Mb:

custom:
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
    packagerOptions:
      scripts:
        - npm rebuild sharp --target=10.15.0 --target_arch=x64 --target_platform=linux

package: 
  exclude:
    - .env
    - .git/**
    - .gitlab-ci.yml
    - tests*
  excludeDevDependencies: true
  individually: true

I've also played around with the package.exclude, but with no luck:

- node_modules/**
- '!node_modules/sharp/**'

and this is my webpack config:

const path = require('path');
const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');

const entries = {};

Object.keys(slsw.lib.entries).forEach(key => (entries[key] = ['./source-map-install.js', slsw.lib.entries[key]]));

module.exports = {
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  entry: slsw.lib.entries,
  devtool: 'source-map',
  resolve: {
    extensions: ['.js', '.jsx', '.json', '.ts', '.tsx'],
  },
  //   externals: ['sharp'], #tried that too
  externals: [nodeExternals()],
  output: {
    libraryTarget: 'commonjs',
    path: path.join(__dirname, '.webpack'),
    filename: '[name].js',
  },
  target: 'node',
  module: {
    rules: [
      // all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
      { test: /\.ts?$/, loader: 'ts-loader', options: { happyPackMode: true } },
    ],
  },
};

When running locally, I can see what it's packaging... the node_modules folder has sharp and its dependencies, it seems. But the biggest folder is sharp.

I suspect I'm packaging stuff inside sharp folder that I don't need... but I can't seem to understand what.

Any help?

Thanks

UPDATE: Reading more carefully, it seems the function where I need sharp (origin-response) size limit is 5Mb. I just need to find a way to package sharp only for that function. Webpack seems to put it in both, even though I don't need it on the other function (viewer request). Any help on this?

noetix · Accepted Answer · 2020-02-10 21:31:05Z

2

I ended up running a script in custom.webpack.packagerOptions.scripts that will ignore sharp where it's not needed.

This is the script I used:

custom:
    webpack:
        includeModules:
            forceExclude:
                - aws-sdk
        packagerOptions: # uncomment this block if invoking locally
            scripts:
                - if [ -f "src/handlers/myfunction.js" ]; then rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp; else rm -rf node_modules; fi

edited Feb 10, 2020 at 21:31

noetix

4,9533 gold badges29 silver badges48 bronze badges

answered Feb 7, 2020 at 10:29

paezinc

3391 gold badge3 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

noetix Over a year ago

Should post the script to do that, I ended up running my build in a docker container to work around this problem.

Collectives™ on Stack Overflow

Use sharp on lambda@edge

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related