AWS Lambda Container works locally but fails when deployed

Question

I wrote a python function for solving a Mixed Integer Linear Program, which uses the Pyomo Library and SCIP as external solver. My goal is to deploy the function as an AWS Lambda Function. Since I also need to install SCIP, I decided to deploy the Lambda as a Docker Container. I do not use an AWS Base Image but rather a custom image (python 3.10-bullseye). My Dockerfile looks very similar to what you find in the AWS Documentation (text), except that I also have a line to install the SCIP package:

RUN apt-get update && apt-get install -y ${FUNCTION_DIR}/SCIPOptSuite-8.0.3-Linux-debian.deb

Now, when I try to test the Lambda function locally, like it is explained in the documentation (using curl -XPOST ...), everything works perfectly fine. However, when I then push my image to the ECR and deploy a Lambda function based on it, it doesn't work anymore.

The concrete Error Message I get is:

START RequestId: 8fe2a5ca-4aa2-4e38-9be1-c2d4b1719147 Version: $LATEST
[ERROR] 2023-07-05T13:32:44.116Z    8fe2a5ca-4aa2-4e38-9be1-c2d4b1719147    Solver (scip) returned non-zero return code (-11)
[ERROR] 2023-07-05T13:32:44.116Z    8fe2a5ca-4aa2-4e38-9be1-c2d4b1719147    See the solver log above for diagnostic information.
[ERROR] ApplicationError: Solver (scip) did not exit normally

and it happens in the line where Pyomo calls the external solver:

solver.solve(model, tee=True).

According to SCIP, error code -11 stands for SCIP_PLUGINNOTFOUND, so it seems like something is missing inside the container, or at least is not where it is expected to be found.

While the error message tells me to see the solver log above, there are no logs that I can see from the solver. I have tried to find out at least which plugin is missing but I failed. I believe that ultimately, it won't come down to which plugin is missing, but rather is an AWS Lambda issue not releated to SCIP directly, so it would be interesting to know what could have gone wrong here, or how I could debug the problem and find out more about it.

Yes, when I test it locally, I have to run the image first before making the POST request. — TechnoX
– TechnoX, Commented Jul 5, 2023 at 14:49
"I do not use an AWS Base Image but rather a custom image" - I assume your custom image includes the lambda runtime "awslambdaric"? — cyberwombat
– cyberwombat, Commented Jul 5, 2023 at 17:25
Exactly. I don't believe the code would run at all otherwise. The error only happens when the code calls the solver. — TechnoX
– TechnoX, Commented Jul 5, 2023 at 17:48

TechnoX · Accepted Answer · 2023-07-06 09:15:04Z

1

I managed to resolve it. When the error message told me that the solver returned exit code -11, I assumed that it really came from the solver. But I found out, that the solver died with a SIGSEGV, which corresponds to signal 11. After all, the problem was just that I had given my Lambda too little memory or storage.

edited Jul 6, 2023 at 9:15

answered Jul 6, 2023 at 9:14

TechnoX

213 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS Lambda Container works locally but fails when deployed

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related