My awscli is not working inside the lambda function

Question

I am trying to run AWS CLI commands on a Lambda function. I referred to How to use AWS CLI within a Lambda function (aws s3 sync from Lambda) :: Ilya Bezdelev and generated a zip file with the awscli package. When I try to run the lambda function I get the following error:

START RequestId: d251660b-4998-4061-8886-67c1ddbbc98c Version: $LATEST
[INFO]  2020-06-22T19:15:45.232Z    d251660b-4998-4061-8886-67c1ddbbc98c

Running shell command: /opt/aws --version

Traceback (most recent call last):
  File "/opt/aws", line 19, in <module>
    import awscli.clidriver

ModuleNotFoundError: No module named 'awscli'

What could be the issue here?

Are you using the lambda layer shown here or you are trying to construct it yourself? — Marcin
– Marcin, Commented Jun 22, 2020 at 21:46
What is your actual goal? What commands are you wanting to run and what result are you seeking? Using awscli from Lambda isn't advisable and there might be an alternative if you can tell us more about your needs. — John Rotenstein
– John Rotenstein, Commented Jun 23, 2020 at 3:12
Marcin, I tried to construct by myself with the instructions from the URL mentioned I generated the zip file with aws package and added it to the layers. When I try to run the script I get this error — Nirja Ayare
– Nirja Ayare, Commented Jun 23, 2020 at 15:04
John Rotenstein, my company does not give access to aws cli to developers and other teams apart from my team(the cloud engineering team) . So, I was trying to find a way by which the developers could access aws cli and came across this method. As they can run lambda functions they could use this way. Hope this clears my goal! — Nirja Ayare
– Nirja Ayare, Commented Jun 29, 2020 at 14:11

Dharman · Accepted Answer · 2020-09-11 19:51:28Z

1

Everything in the 'site-packages' folder needs to be directly in the zip, and subsequently the /opt/ folder for the lambda, NOT nested inside a 'site-packages' folder which is what the tutorial results in unfortunately when you use his commands verbatim.

edited Sep 11, 2020 at 19:51

Dharman♦

33.9k27 gold badges106 silver badges157 bronze badges

answered Sep 11, 2020 at 18:05

tvmaynard

111 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

Abdel-Raouf · Accepted Answer · 2022-07-29 13:56:12Z

AS @tvmaynard said, you first need to add all the packages inside the same path as aws script of the AWS-CLI, by using this command:

cp -r ../${VIRTUAL_ENV_DIR}/lib/python${PYTHON_VERSION}/site-packages/. .

But, Even after that you will face a problem that there is some libraries that AWS-CLI is dependent on and must be installed in the Runtime Python as PyYAML, to install it you need to have access to Python Runtime inside the lambda, which is Not allowed.

Even if, you try to solve this by telling the interpreter where to search for the PyYAML library and installed it inside /tmp/, as follow:

import sys
import subprocess
stderr = subprocess.PIPE
stdout = subprocess.PIPE

cmd_1 = "pip install PyYAML -t /tmp/ --no-cache-dir"
subprocess.Popen(
    args=cmd_1, start_new_session=True, shell=True, text=True, cwd=None, 
 stdout=stdout, stderr=stderr
    )

 sys.path.insert(1, '/tmp/')

 import yaml

You will be able to use the library by importing it only inside the lambda function context as if you added a layer to the lambda, But not from the underlying command line which is linked to the python interpreter inside the lambda Runtime and have a default path to search for the libraries it needs.

You will also, Pay More Money to install this and the may needed other libraries, every time you trigger your lambda, Which if on Production ENV, will add more money to your Bill, that doesn't generate value.

Collectives™ on Stack Overflow

My awscli is not working inside the lambda function

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related