7

Consider the following pseudo code:

def g_user():
    while True:
        yield read_user_input()

def g_socket():
    while True:
        yield read_socket_input()

def g_combined(gu, gs):
    # should read user or socket input, whichever is available

    while True:
        sel = select(gu, gs)
        if sel.contains(gu):
            yield gu.next()
        if sel.contains(gs):
            yield gs.next()

gc = g_combined ( g_user(), g_socket() )

How to implement this in easiest way?

3
  • 1
    I'm not sure, but I think you might need to use an object more complex than just a generator, since besides just implementing __next__(), you'll need some kind of is_ready() method. Commented Feb 8, 2012 at 1:46
  • How to make this more complex object? Why it is not in standard library? (expected to see such thing in itertools, like itertools.multiplex) Commented Feb 8, 2012 at 1:52
  • Its probably not in itertools for the reason explained by your reference to genmulti. It requires threading to be accomplished since the generators don't really have a polling mechanism. Commented Feb 8, 2012 at 2:31

3 Answers 3

8

Look's like someone already implemented this: http://www.dabeaz.com/generators/genmulti.py

Mirrored here:

import Queue, threading
def gen_multiplex(genlist):
    item_q = Queue.Queue()
    def run_one(source):
        for item in source: item_q.put(item)

    def run_all():
        thrlist = []
        for source in genlist:
            t = threading.Thread(target=run_one,args=(source,))
            t.start()
            thrlist.append(t)
        for t in thrlist: t.join()
        item_q.put(StopIteration)

    threading.Thread(target=run_all).start()
    while True:
        item = item_q.get()
        if item is StopIteration: return
        yield item
Sign up to request clarification or add additional context in comments.

Comments

1

In order to make select.select() work properly, you'll need to expose a fileno() method on your generators that returns the underlying file descriptor.

1 Comment

"select" is just for pseudocode, to show what exactly I want.
-2

itertools module from standard library contains nice functions. In this specific case cycle function is good.

cycle help says:

cycle(iterable) --> cycle object

Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely.

Using cycle in your solution:

import itertools

def g_user():
    while True:
        yield read_user_input()

def g_socket():
    while True:
        yield read_socket_input()


def g_combine(*generators):
    for generator in itertools.cycle(generators):
        yield generator.next()


g_combined =  g_combine(g_user(), g_socket())

Considerations about this code:

  1. g_combine ends with first generator that raise StopIteration

  2. g_combine accepts N generators

  3. How catch exceptions correctly? Think about.

1 Comment

No, it should not just interleave them like in "AAA", "BBB", "CCC" -> "ABCABCABC". The order in which input generators becomes readable is not predictable. It should not wait for g_user when g_socket is readable.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.