AWS Lambda fails when using Playwright to navigate browser (Python)

Question

I have created an LLM agent with tools using Langchain and Python. I build my Lambda with with AWS SAM with Docker. After building, I test the Docker image using sam local invoke, and the Lambda function executes as expected (on M1 Max chip), using Langchain's Playwright toolkit to navigate a headless, asynchronous chromium browser. After I deploy with SAM, I test the function in the AWS Lambda UI, but there is an issue with navigating the browser: Target page, context or browser has been closed. I do not experience this error when testing the lambda locally with sam local invoke.

Additionally, if the LLM agent does not use Playwright tools, then the Lambda executes as expected. This confirms that there is an issue using Playwright (via the Langchain package) within a deployed AWS Lambda function that is not experienced with local testing.

Used image with Playwright in Dockerfile: mcr.microsoft.com/playwright/python:v1.21.0-focal. Expected Playwright browsers to properly configured. Browser still closes prematurely.
Increased to 2048 MB. I hoped that more memory might solve the issue, but error still occurs.
Followed this guide for Dockerfile. I thought by adding aws-lambda-cpp build dependencies the issue might resolve, but it did not.
Tried --platform=linux/x86_64 and --platform=linux/arm64 with Playwright image. Architecture set in Docker to correspond to choice. Error still occurs.

ggorlen · Accepted Answer · 2023-11-09 18:35:27Z

The issue was related to Langchain's initialization of the async Playwright browser. The imported function from langchain.tools.playwright.utils import create_async_playwright_browser does not allow for arguments to be passed to browser.chromium.launch(). For the browser to function properly on AWS, browser.chromium.launch(headless=True, args=["--disable-gpu", "--single-process"]).

Here is the modification I made to get the Playwright browser to function properly in my AWS Lambda:

from playwright.async_api import async_playwright
from playwright.async_api import Browser as AsyncBrowser
from typing import List
from langchain.tools.playwright.utils import run_async

def create_async_playwright_browser(headless: bool = True, args: List[str] = None) -> AsyncBrowser:
    """
    Create an async playwright browser.

    Args:
        headless: Whether to run the browser in headless mode. Defaults to True.
        args: A list of arguments for the browser instance.

    Returns:
        AsyncBrowser: The playwright browser.
    """
    browser = run_async(async_playwright().start())
    if args:
        return run_async(browser.chromium.launch(headless=headless, args=args))
    else:
        return run_async(browser.chromium.launch(headless=headless))

Using this function to create the browser instead of the one found langchain.tools.playwright.utils results in the AWS Lambda functioning as desired

Collectives™ on Stack Overflow

AWS Lambda fails when using Playwright to navigate browser (Python)

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related