Python Flask App Freezes with Large Data: Seeking Optimization Advice

Question

I'm developing a Python Flask application to find combinations of numbers from dataset that sum up to a target value. The app works fine with smaller datasets but freezes when processing larger datasets. Here GitHub repository of my code: GitHub

I have to admit that I have no prior knowledge of coding and rely almost entirely on ChatGPT. But it can't solve this issue

This is my online web app that I hosted on pythonanywhere.com find_combi

you can see that I have to put a warning '[RECOMMEND Maximum: 5 values]' on. if input exceed this it will likely freeze, and user cannot just refresh the page. I have to go to reload on pythonanywhere.com myself.

input

result

The application freezes completely with larger dataset. I'm okay with longer load times as long as the application doesn't freeze completely

The goal is to make it able to accept around 100 input and not freezes

I realized that you're not using pandas for data processing. Before using celery or other async librarires I'd suggest you to use pandas or numpy. I bet this will improve your response time. With pandas you can "merge" dataframes as well. Hope this comment can help you — Víctor Cortés
– Víctor Cortés, Commented Aug 3, 2024 at 6:09
Please include a reproducible example in your question and not use images for code and errors. — oskar
– oskar, Commented Aug 4, 2024 at 10:19

Farrukh Naveed Anjum · Accepted Answer · 2024-08-03 05:21:01Z

0

For long running jobs you can need to use the background processing. You can use Celery for this particular purpose. Otherwise you will up in hanging applicaiton like you are currently facing.

I suggest you to have a look at the following Offical Documentation of Python Celery Package. https://docs.celeryq.dev/en/stable/getting-started/introduction.html

Using Celery you can achieve the ability to scaleup your Flask Application. Plus kindly ensure the proper resources for your Flask Application. In my experience PythonAnywhere does not scale well.

answered Aug 3, 2024 at 5:21

Farrukh Naveed Anjum

2503 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Flask App Freezes with Large Data: Seeking Optimization Advice

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related