Most efficient way to dequeue bytes from Queue in C#

Question

I have a Queue with some data. If I want to retrieve 5 bytes (or as much as is available if less than 5), which of the following methods is more efficient, or does it make no difference?

Option 1:

        byte[] ret = null;

        if (this.dataQueue.Count >= numBytes)
        {
            ret = new byte[numBytes];
        }
        else
        {
            ret = new byte[this.dataQueue.Count];
        }

        for (int i = 0; i < ret.Length; i++)
        {
            ret[i] = this.dataQueue.Dequeue();
        }

        return ret;

Option 2:

        List<byte> l = new List<byte>();

        try
        {
            for (int i = 0; i < numBytes; i++)
            {
                l.Add(this.dataQueue.Dequeue());
            }
        }
        catch
        {
            System.Diagnostics.Debug.WriteLine(string.Format("Unable to retrieve requested number of bytes from queue. ({0} bytes requested, {1} bytes available)", numBytes, l.Count));
        }

        return l.ToArray();

Thanks in advance for any advice.

Are you actually having performance problems that you are trying to solve? If so, did you identify this as the bottleneck? — Oded
– Oded, Commented Nov 9, 2011 at 11:05
When you say "queue", what specifically are you talking about? What kind of datastructure are you using? — Lasse V. Karlsen
– Lasse V. Karlsen, Commented Nov 9, 2011 at 11:10
I have not yet performed any tests.I have not seen any bottleneck. I just realised that there were two ways to perform the same task and was curious more than anything. I am using a System.Collections.Generic.Queue<byte> — Mark
– Mark, Commented Nov 9, 2011 at 14:21

Grant Thomas · Accepted Answer · 2011-11-09 11:35:23Z

1

Performance really depends on your scenario, and always is the case that someone will warn you not to optimise prematurely and find the bottlenecks before assuming something is slow and such and so, and then there are also presumptions from angles that may or may not hold true; I really align with the former part of that, but don't like to jump to conclusions, and think the parts of a program are demonstrated to be slower and hence call out for the need for such as part to be optimised.

With that, anyway, consider the following, primitive, not entirely practical ad hoc program (using your methods, named Dequeue and TryDequeue here, but not listed):

var data = new byte[4096];
var random = new Random();      
var queue = new Queue<byte>();
var stopWatch = new Stopwatch();

TimeSpan dequeueTimespan, tryDequeueTimespan;

stopWatch.Restart();
for (int i = 0; i < 1000000; i++)
{
    random.NextBytes(data);
    foreach (var item in data)
    {
        queue.Enqueue(item);
    }
    Dequeue(queue, 1024);
    queue.Clear();
}
stopWatch.Stop();
dequeueTimespan = stopWatch.Elapsed;

stopWatch.Restart();
for (int i = 0; i < 1000000; i++)
{
    random.NextBytes(data);
    foreach (var item in data)
    {
        queue.Enqueue(item);
    }
    TryDequeue(queue, 1024);
    queue.Clear();
}
stopWatch.Stop();
tryDequeueTimespan = stopWatch.Elapsed;

Console.WriteLine("Dequeue:    {0}", dequeueTimespan);
Console.WriteLine("TryDequeue: {0}", tryDequeueTimespan);
Console.ReadKey();

Now, if we run that over 10000 iterations for each, we get:

Dequeue:    00:00:12.6178244
TryDequeue: 00:00:13.6747715

And for 1000000 iterations per method:

Dequeue:    00:02:16.4144764
TryDequeue: 00:02:13.2039597

Granted, we're doing other work here, but it is all relative; we're also exaggerating your case here, as you only require 5 bytes, so...

In the first case of 10000 iterations again:

Dequeue:    00:00:10.5624014
TryDequeue: 00:00:10.2529997

I'm not saying these are perfect, results, but they are results; But the point is this.

That the performance degradation is negligible between the two.

There are also many other factors to consider in a practical environment that, needless to say and not to try and list all variabilities here, you will be (or become) aware of and ought to take the time to tie in to your scenario.

edited Nov 9, 2011 at 11:35

answered Nov 9, 2011 at 11:28

Grant Thomas

45.1k10 gold badges88 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mark Over a year ago

Hi, thanks for such a detailed reply! I was expecting a greater difference, since I was not sure if List<> preallocates space or if it would need to perform significant work on every Add. I was also wondering if try/catch blocks are expensive. But with such a negligible difference, is there either one which would be considered better code?

Bartosz Over a year ago

also remember that allocations are relatively cheap, but if you create lots of garbage, your collection cycles may be much longer

Neowizard · Accepted Answer · 2011-11-09 15:41:39Z

0

Don't take this the wrong way, but you're asking an almost impossible question. When dealing with such a small difference, you need a whole lot more information to make a ruling either way.

For instance, what optimizations are in place, and which architecture is this going to run on. Moreover, what's the current program state & flow (is memory tight? is the CPU working at near 100%?), and what about caching (applicative\OS\hardware).

This list is not complete, is as far from it, but it should pass the point - if you're working with optimizations and performance on a pin-point scale, you're going to need more detail (not only for us, but for anyone who might have an idea for an answer).

If you can expend the details of you're question it'll be wonderful.

EDIT:

Note that the two methods dequeue the Queue in the same way (A for-loop with a call to Dequeue()), and so are not different in this regard, but rather in the way they collect the items dequeued from the Queue. If the question is what the fastest way to do this (Use the dequeued items), it's simply not to collect them and send them on their flow. If the items are meant to be returned then you go back to the original question, but with a sub-question that might help you - What do you expect to be done with the collection of items after they're dequeued.

Hope this helps

edited Nov 9, 2011 at 15:41

answered Nov 9, 2011 at 11:35

Neowizard

3,0571 gold badge24 silver badges42 bronze badges

6 Comments

Mark Over a year ago

Hi, thanks for the reply. I had not considered all of those factors, since I was assuming that whatever platform, there would be one method which is more efficient relative to the other. My question was more for curiosity, since I would like the library to be quite portable and so should not have just high dependence on such details. If that makes it impossible to pick a preferable solution, then not to worry. Thanks for your help.

Neowizard Over a year ago

In this case, I'll just note that the methods are do not differ in the way they dequeue the items (in most regards). I've edited the replay to explain

Mark Over a year ago

Thanks for the update. The bytes returned are sent as an array of bytes to another library, hence needing them in that format at the end.

Neowizard Over a year ago

If they're going to be used as an array in the end, you'd do best to start with an array to begin with (if performance is the issue at hand)

Mark Over a year ago

Makes sense. I figured as much, but wasn't sure if it was a big difference - or if infact it was doing similar work behind the scenes for each approach. Thanks for the clarification.

|

Bartosz · Accepted Answer · 2011-11-09 16:46:03Z

0

The first, because it does 1 allocation, while

l.Add(this.dataQueue.Dequeue())

need to reallocate the space needed for list few times, unless you reserve it up front:

l.Capacity = 5

which does not affect your list length, it just reserves memory space for elements (and we know you will have at most 5).

Of course it's also worth to check if you really need to have bytes in queue and call Dequeue each time you need to fetch one byte.

edited Nov 9, 2011 at 16:46

answered Nov 9, 2011 at 11:06

Bartosz

3,38824 silver badges31 bronze badges

4 Comments

Mark Over a year ago

Thanks for the reply. I have never seen Reserve before and cannot find any reference on the net. Do you have a link to the documentation for this method? One of the reasons for asking this question originally was that I was unsure if List<> needs to reallocate space every time, or if it is more intelligent and so when created, takes a set amount of space for use (because you can specify a capacity). So thanks for clarifying this.

Bartosz Over a year ago

Ah sorry I've confused it with another language/class. It's Capacity property. Will update my original answer.

Mark Over a year ago

Strange. I just replied but it has disappeared, so sorry if you get this twice! But thanks for the update. That makes more sense. So if the capacity is already great enough, does this still have a significant effect?

Bartosz Over a year ago

Should be about the same, unless you are calling Add much, much more than five times:)

Collectives™ on Stack Overflow

Most efficient way to dequeue bytes from Queue in C#

3 Answers 3

2 Comments

6 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

6 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related