How to use REQ and REP in pyzmq with asyncio?

Question

I'm trying to implement asynchronous client and server using pyzmq and asyncio in python3.5. I've used the asyncio libraries provided by zmq. Below is my code for client(requester.py) and server(responder.py). My requirement is to use only REQ and REP zmq sockets to achieve async client-server.

requester.py:

import asyncio
import zmq
import zmq.asyncio

async def receive():
        message = await socket.recv()
        print("Received reply ", "[", message, "]")
        return message

async def send(i):
        print("Sending request ", i,"...")
        request = "Hello:" + str(i)
        await socket.send(request.encode('utf-8'))
        print("sent:",i)

async def main_loop_num(i):
        await send(i)
        #  Get the reply.
        message = await receive()
        print("Message :", message)

async def main():
        await asyncio.gather(*(main_loop_num(i) for i in range(1,10)))                

port = 5556
context = zmq.asyncio.Context.instance()

socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:%d" % port)
asyncio.get_event_loop().run_until_complete(asyncio.wait([main()]))

responder.py:

import asyncio
import zmq
import zmq.asyncio

async def receive():
    message = await socket.recv()
    print("Received message:", message)
    await asyncio.sleep(10)
    print("Sleep complete")
    return message

async def main_loop():
    while True:
        message = await receive()
        print("back to main loop")
        await socket.send(("World from %d" % port).encode('utf-8'))
        print("sent back")

port = 5556
context = zmq.asyncio.Context.instance()

socket = context.socket(zmq.REP)
socket.bind("tcp://*:%d" % port)
asyncio.get_event_loop().run_until_complete(asyncio.wait([main_loop()]))

The output that I'm getting is:

requester.py:

Sending request  5 ...
sent: 5
Sending request  6 ...
Sending request  1 ...
Sending request  7 ...
Sending request  2 ...
Sending request  8 ...
Sending request  3 ...
Sending request  9 ...
Sending request  4 ...

responder.py:

Received message: b'Hello:5'
Sleep complete
back to main loop
sent back

From the output, I assume that the requester has sent multiple requests, but only the first one has reached the responder. Also, the response sent by responder for the first request has not even reached back to the requester. Why does this happen? I have used async methods everywhere possible, still the send() and recv() methods are not behaving asynchronously. Is it possible to make async req-rep without using any other sockets like router, dealer, etc?

Cloudomation · Accepted Answer · 2019-11-18 10:58:54Z

3

ZMQs REQ-REP sockets expect a strict order of one request - one reply - one request - one reply - ...

your requester.py starts all 10 requests in parallel:

await asyncio.gather(*(main_loop_num(i) for i in range(1,10)))

when sending the second request ZMQ complains about this:

zmq.error.ZMQError: Operation cannot be accomplished in current state

Try to change your main function to send one request at a time:

async def main():
    for i in range(1, 10):
        await main_loop_num(i)

If you need to send several requests in parallel then you can't use a REQ-REP socket pair but for example a DEALER-REP socket pair.

answered Nov 18, 2019 at 10:58

Cloudomation

1,6821 gold badge8 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to use REQ and REP in pyzmq with asyncio?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related