Making connections with python3 asyncio

Question

I'm trying to connect to more than one server at the same time. I am currently using loop.create_connection but it freezes up at the first non-responding server.

gsock = loop.create_connection(lambda: opensock(sid), server, port)
transport, protocol = loop.run_until_complete(gsock)

I tried threading this but it created problems with the sid value being used as well as various errors such as RuntimeError: Event loop is running and RuntimeError: Event loop stopped before Future completed. Also, according my variables (tho were getting mixed up) the protocol's connection_made() method gets executed when transport, protocol = loop.run_until_complete(gsock) throws an exception.

I don't understand much about the asyncio module so please be as thorough as possible. I dont think I need reader/writer variables, as the reading should be done automatically and trigger data_received() method.

Thank You.

dano · Accepted Answer · 2015-05-27 15:53:21Z

3

You can connect to many servers at the same time by scheduling all the coroutines concurrently, rather than using loop.run_until_complete to make each connection individually. One way to do that is to use asyncio.gather to schedule them all and wait for each to finish:

import asyncio

# define opensock somewhere

@asyncio.coroutine
def connect_serv(server, port):
    try:
        transport, protocol = yield from loop.create_connection(lambda: opensock(sid), server, port)
    except Exception:
        print("Connection to {}:{} failed".format(server, port))

loop = asyncio.get_event_loop()
loop.run_until_complete(
    asyncio.gather(
      connect_serv('1.2.3.4', 3333),
      connect_serv('2.3.4.5', 5555),
      connect_serv('google.com', 80),
 ))
loop.run_forever()

This will kick off all three coroutines listed in the call to gather concurrently, so that if one of them hangs, the others won't be affected; they'll be able to carry on with their work while the other connection hangs. Then, if all of them complete, loop.run_forever() gets executed, which will allow you program to continue running until you stop the loop or kill the program.

The reader/writer variables you mentioned would only be relevant if you used asyncio.open_connection to connect to the servers, rather than create_connection. It uses the Stream API, which is a higher-level API than the protocol/transport-based API that create_connection uses. It's really up to you to decide which you prefer to use. There are examples of both in the asyncio docs, if you want to see a comparison.

edited May 27, 2015 at 15:53

answered May 27, 2015 at 15:36

dano

95.5k21 gold badges234 silver badges231 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

baudsmoke Over a year ago

Im still wondering if its okay to use run_until_complete() while run_forever() is going. I guess I'll try that after I rewrite.

baudsmoke Over a year ago

Actually this is not a working solution because the sockets are not fully active (unless all the server connections succeed in a timely manner). The program hangs waiting for one of the connections to a server.

dano Over a year ago

@baudsmoke I'm not sure what you mean. What sockets aren't fully active? What specifically is hanging? The program is designed to block in the run_until_complete call, possibly forever if one of the calls to connect_serv never returns. However, the connect_serv calls that don't hang should continue to work just fine.

Collectives™ on Stack Overflow

Making connections with python3 asyncio

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related