python - How to view Entire Queue Contents & Automatic Queue Pushout

Question

I am writing a script that uses a queue to track the last six numbers of a list of length N. In other words, as I iterate over the list of length N, I want to track the last six numbers. I thought a queue would be a good structure to do this because it would automatically push out the last 6th number as I push in a new number. My code is below:

if __name__ == '__main__':

    data = [1,23,45,5,43,2,54,2,34,32]
    scub = DataScrubber(data)
    scub.scrub_zeros(0.01)

    print "TEST"


def scrub_zeros(self,upperzero_range):
        scrubbed_data = [] 
        last_6_data_points = Queue.Queue(6)

        for data in self.data:
            print last_6_data_points
            print scrubbed_data
            last_6_data_points.put(data)
            scrubbed_data.append(data)  
        self.data = scrubbed_data

When I run this script I get the following output:

<Queue.Queue instance at 0x03E25D00>
[]
<Queue.Queue instance at 0x03E25D00>
[1]
<Queue.Queue instance at 0x03E25D00>
[1, 23]
<Queue.Queue instance at 0x03E25D00>
[1, 23, 45]
<Queue.Queue instance at 0x03E25D00>
[1, 23, 45, 5]
<Queue.Queue instance at 0x03E25D00>
[1, 23, 45, 5, 43]
<Queue.Queue instance at 0x03E25D00>
[1, 23, 45, 5, 43, 2]

However, this script never stops running. It freezes and I have to break out using ctrl+break. So after thinking, I figured out the reason why it is freezing. It freezes because when I "put" data into the queue, is doesn't automatically push ot the older data at the other end of the queue. So I guess that the queue is stuck. Am I missing something, is that not suppose to be the point of how a queue works? Do I have to call "get"? Is there anyway around having to call another function for the queue, where it will automatically push out the oldest data when new data is pushed into the queue?

My second question:

How can I view the entire queue like a list? So:

print my_queue

Results in:

[null,1, 23, 45, 5, 43]

after pushing in 5 numbers into 6 line queue.

ShadowRanger · Accepted Answer · 2015-09-29 14:10:07Z

2

Queues don't evict entries when they hit their maximum size, they block when you try to put an entry in that would exceed their maximum size (that's why your program stalls). They're designed for atomic interthread communication, so viewing their contents isn't a design goal, it's to allow get-ing and put-ing in a thread-safe way (with blocking when appropriate).

Since you're not using the interthread hand-off with blocking features of Queue, you don't really want a Queue at all, you want collections.deque with a maxlen.

from collections import deque

...

def scrub_zeros(self,upperzero_range):
    scrubbed_data = [] 
    last_6_data_points = deque([None]*6, maxlen=6)

    for data in self.data:
        print last_6_data_points
        print scrubbed_data
        last_6_data_points.append(data)
        scrubbed_data.append(data)  
    self.data = scrubbed_data

The stringification of a deque will show the elements but it will include extra kruft; if you want it to look list-like w/o the kruft, change the print to print list(last_6_data_points).

answered Sep 29, 2015 at 14:10

ShadowRanger

158k12 gold badges221 silver badges315 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

ShadowRanger Over a year ago

This will print None, not null for the unfilled entries, but I'm assuming that's okay. You could change it to the str, "null" if you want (though that will print with quotes around it); if that's not acceptable, you'd need to format the output yourself instead of relying of the list repr to do it for you.

bud Over a year ago

null was pseudo code. I will take a closer look at your code. Thanks.

ShadowRanger Over a year ago

@budder: Okay. Usually, I'd omit the initializer and just have a maxlen, because having filler entries doesn't really gain you anything, but your example showed filler so I provided it. If you don't need the filler, just initialize with: last_6_data_points = deque(maxlen=6)

bud Over a year ago

Reviewed your code. It works well however I would make one modification. Eventually I will need to access the last_6_data_points and therefore, I want the actual list of data: list(last_6_data_points).

ShadowRanger Over a year ago

@budder: You can access the values from last_6_data_points without converting to a list. deques iterate the same as a list, for item in mydeque: works the same as for item in mylist: (because they're both sequences). Since the deque is so short, it doesn't even have the negatives normally associated with deques (that indexing near the middle of the deque is linear access time), so you can index it as needed, mydeque[4] for a short deque will be just fine. Converting back to a list isn't bad, it's just not necessary to access the data; deques give full access.

Collectives™ on Stack Overflow

python - How to view Entire Queue Contents & Automatic Queue Pushout

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related