Upload code to AWS lambda error Runtime.ImportModuleError

Question

I am trying to put this scraping code on AWS Lambda, when i try to test the code i get the following error:

"errorMessage": "Unable to import module 'lambda_function': No module named 'lambda_function'",
  "errorType": "Runtime.ImportModuleError"

The code:

from datetime import datetime
from functions import MultiplePageScraper, CleanTable, upload_unique, pages
from sqlalchemy import create_engine
now = datetime.now()
date = now.date()

# Credentials:
host = xxxxx
user = "admin"
password = xxxxx
port = 3306
database = "vehicles"
# Create connection
mydb = create_engine("mysql+pymysql://" + user + ':' + password + '@' + host + ':' + str(port) + '/' + database , echo=False)

# Target Url :
URL = "https://www.usedcars.co.ke/nairobi/cars-for-sale?page="

def lambda_handler(event, context):
    # Retrive the data and clean it
    page_count = pages(URL)
    data = MultiplePageScraper(URL, int(page_count))
    clean_data = CleanTable(data)
    # Check for duplicates scraped Vs Database
    upload_data = upload_unique(clean_data, mydb)
    # Upload data to RDS database
    if upload_data.shape[0] > 1:
        upload_data.to_sql(name='CARS', con=mydb, if_exists = 'append', index=False)
        print(f'Uploaded {upload_data.shape[0]} unique rows to the database!')
    else:
        print('No new cars to upload.')

I have no idea what the error means, any one can help?

You need to update the handler in the Runtime settings for the Lambda function. For example: if the name of the python module for the lambda is cars.py, set the handler in Runtime settings to cars.lambda_handler. — Oluwafemi Sule
– Oluwafemi Sule, Commented Mar 21, 2021 at 16:34
@OluwafemiSule Thanks for the tip. It worked. Now after packaging up the code with all dependencies and uploading it to lambda i get the following error: "errorMessage": "Unable to import module 'Get_Data': cannot import name 'etree' from 'lxml' (/var/task/lxml/__init__.py)", "errorType": "Runtime.ImportModuleError" Any ideas what might be the problem ? — Youssef Razak
– Youssef Razak, Commented Mar 22, 2021 at 10:02

Marcin · Accepted Answer · 2021-03-21 23:20:04Z

3

By default, a new python lambda function will look for lambda_function.py file and call lambda_handler method:

You can change it using the runtime settings for your function.

answered Mar 21, 2021 at 23:20

Marcin

241k16 gold badges315 silver badges368 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Youssef Razak Over a year ago

@ Marcin thanks. Now after packaging up the code with all dependencies and uploading it to lambda i get the following error: "errorMessage": "Unable to import module 'Get_Data': cannot import name 'etree' from 'lxml' (/var/task/lxml/__init__.py)", "errorType": "Runtime.ImportModuleError" Any ideas what might be the problem ?

Marcin Over a year ago

@YussufAbdirizak Is this some custom python module? You can create new question specific to this new issue.

Guillermo Over a year ago

Make sure that you put the library that you import for Python inside the /python folder.

Collectives™ on Stack Overflow

Upload code to AWS lambda error Runtime.ImportModuleError

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related