1

I spun up a cloud server and put my flask web app code plus a frontend in there. When a user runs an operation on the frontend this gets sent to the backend endpoint as a JSON, then processed and another JSON gets sent back to the frontend. I want to say that I am new to this and seeking advice to best handle situations like 100+ simultaneous users.

I can imagine one user making a call to the flask endpoint, that gets processed and sent back to the frontend - but what if 100+ users are hitting this same endpoint? Would it be a first come first serve situation, like a queue? But what about the possibility of two people hitting the endpoint at the exact same time? I am using Gunicorn instead of the default flask server (via flask run).

I am new to these concepts, but I am wondering how these might apply (if at all ) to my situation and if so how I'd go about integrating them into my code/infrastructure:

  1. Queuing
  2. Caching
  3. Asychronous
  4. Multi-processing
  5. Multi-threading
  6. Concurrency
  7. Parallelism
  8. Distributed systems

If anyone could please add clarity it would be greatly appreciated. If there are any monitoring tools I could attach to this application that would also be great.

Thank you

Improve this question
10
  • « But what about the possibility of two people hitting the endpoint at the exact same time » : unless you have two network cards, and two completely separated route, with no common routers in between, that can't really happen (and even in that case, it doesn't matter. One packet will be before the other). Commented Mar 23 at 16:09
  • For the rest, your interrogation is a valid interrogation. But it is hard to give an answer with such a vague description and no code. I mean, for most of the web application, this is irrelevant. Because flask route are almost instantaneous. So each user get served one after the other. You could have more that one flask server handling in parallel some requests, sure (this is the case by default in most production server). But even if you haven't, like in debug mode, most of the time, that is not a problem. Commented Mar 23 at 16:12
  • There is already an implicit queue, not at the application (flask) level, but at the network and transport level. When the flask server "accept" an incoming connection, it takes that incoming connection from a queue at network level. Even it it can handle only one request at the time, if the request is quickly served, as it should, it has the time to accept, read the request, process the request, write the answer, and go back to accepting another connection. As long as all this occurs under the timeout threshold (which is in the order of magnitude of minutes, not milliseconds) the waiting Commented Mar 23 at 16:15
  • connections will still be there waiting. Commented Mar 23 at 16:15
  • Of course, if processing a request takes a lot of time (for example because it does a lot of heavy computation. Like a route /computeZillionsPiDecimal?numberOfZillions=, then, indeed, if your server accept only one connection at a time, some client would get a timeout). But that would be bad practice. If your route doesn't take way less that 1 second to execute, then you should have some sort of background server-side works in place. I won't expand on that subject, since I have no reason to think that you are in this case. Commented Mar 23 at 16:19
Related questions
6742
How do I return the response from an asynchronous call?
3982
How do I get the current time in Python?
3956
How can I catch multiple exceptions in one line? (in the "except" block)
Load 6 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.