33

Note: this is not about multi threading or multi processing. This question is regarding a single process and single thread.

Python async.io and JavaScript async both are single thread concepts.

In python, async.io, we can use async await keywords to create a function so that when this function is invoked multiple times (via gather) they get executed concurrently. The way it works is that when an await keyword is encountered, other tasks can execute. As explained here we apply the async await keywords to function that we would like to execute concurrently. However while these tasks are running concurrently, the main thread is blocked.

In JavaScript async has evolved from callbacks, promise, async/await. In the main program, when async is encountered, then the function is sent to the event loop (where the function execution begins) and the main thread can continue working. Any subsequent async function also gets added to the event loop. Inside the event loop when the function execution encountered an await then other function is given a chance to execute untill await in encountered.

To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing. Because once we start the child thread/process, and untill we call .join the main thread is not blocked.

Is there anyway by which the python's async.io can make the main thread non blocking? If not, then is this the fundamental difference between async concept in JavaScript and python?

0

1 Answer 1

35
+200

when async is encountered, then the function is sent to the event loop and the main thread can continue working.

This is close, but not quite right. In Javascript, execution won't stop until the callstack has been emptied - the await keyword will suspend the execution of a particular function until an event triggers, and in the mean time, control returns to its caller. This means the first part of any async function will execute as soon as it is called (it's not immediately put into the event loop), and will only pause as soon as an await is hit.

To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing.

The difference here is that by default, Javascript always has an event loop and python does not. In other words, python has an on/off switch for asynchronous programming while Javascript does not. When you run something such as loop.run_forever(), you're basically flipping the event loop on, and execution won't continue where you left off until the event loop gets turned back off. (calling it a "thread" isn't quite the right word here, as it's all single-threaded, as you already acknowledged. Instead, we generally call each task that we queue up, well, a "task")

You're asking if there's a way to let your code continue execution after starting up the event loop. I'm pretty sure the answer is no, nor should it be needed. Whatever you want to execute after the event loop has started can just be executed within the event loop.

If you want your python program to act more like Javascript, then the first thing you do can be to start up an event loop, and then any further logic can be placed within the first task that the event loop executes. In Javascript, this boiler plate essentially happens for you, and your source code is effectively that first task that's queued up in the event loop.

Update:

Because there seems to be some confusion with how the Javascript event loop works, I'll try to explain it a little further.

Remember that an event loop is simply a system where, when certain events happen, a block of synchronous code can be queued up to run as soon as the thread is not busy.

So let's see what the event loop does for a simple program like this:

// This async function will resolve
// after the number of ms provided has passed
const wait = ms => { ... }

async function main() {
  console.log(2)
  await wait(100)
  console.log(4)
}

console.log(1)
main()
console.log(3)

When Javascript begins executing the above program, it'll begin with a single task queued up in it's "run these things when you're not busy" queue. This item is the whole program.

So, it'll start at the top, defining whatever needs to be defined, executes console.log(1), call the main function, enters into it and runs console.log(2), calls wait() which will conceptually cause a background timer to start, wait() will return a promise which we then await, at which point we immediately go back to the caller of main, main wasn't awaited so execution continues to console.log(3), until we finally finish at the end of the file. That whole path (from defining functions to console.log(3)) is a single, non-interruptible task. Even if another task got queued up, Javascript wouldn't stop to handle that task until it finished this chunk of synchronous logic.

Later on, our countdown timer will finish, and another task will go into our queue, which will cause our main() function to continue execution. The same logic as before applies here - our execution path could enter and exit other async functions, and will only stop when it reaches the end of, in this case, the main function (even hitting an await keywords doesn't actually make this line of syncrounous logic stop, it just makes it jump back to the caller). The execution of a single task doesn't stop until the callstack has been emptied, and when execution is continuing from an async function, the first entry of the callstack starts at that particular async function.

Python's async/await follows these same rules, except for the fact that in Python, the event loop isn't running by default.

Sign up to request clarification or add additional context in comments.

12 Comments

I have updated javascript section in my question. Plese can you have a look. Also, originally, in the JavaScript section, I never said that execution will pause until the await keyword is encountered.
So, the "main" program in Javascript doesn't actually have any special treatment. It's just another task in the event loop. The first line you execute in Javascript would be equivalent to the first line of the first task of a python event loop.
@variable I updated my answer to try and explain the event loop a little clearer. Hopefully, this can help with some of the confusion.
Thank you for this. In js, after putting wait in the event loop, control comes back to main. Ok. Question 1: Then when does control goes back to console.log(4)? Is it either after finishing main thread or encountering async keyword in another subsequent function or is there any other possibility? Question 2: can we write await keyword in main program to force control to go to next task?
1. The main (initial) task will run to completion and can't be interrupted, as is the case with any other task. The task to finish executing the main() function can happen as soon as the countdown has finished and no other task is currently running. This could be after main() has finished and between awaits of other function calls. 2. Effectivly yes (assuming all the callers in this callstack are also awaiting this promise).
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.