7

I want to write something like this to a file:

FileStream output = new FileStream("test.bin", FileMode.Create, FileAccess.ReadWrite);
BinaryWriter binWtr = new BinaryWriter(output);
double [] a = new double [1000000];  //this array fill complete

for(int i = 0; i < 1000000; i++)
{
    binWtr.Write(a[i]);
}

And unfortunately this code's process last very long! (in this example about 10 seconds!)

The file format is binary.

How can I make that faster?

0

2 Answers 2

12

You should be able to speed up the process by converting your array of doubles to an array of bytes, and then write the bytes in one shot.

This answer shows how to do the conversion (the code below comes from that answer):

static byte[] GetBytes(double[] values) {
    var result = new byte[values.Length * sizeof(double)];
    Buffer.BlockCopy(values, 0, result, 0, result.Length);
    return result;
}

With the array of bytes in hand, you can call Write that takes an array of bytes:

var byteBuf = GetBytes(a);
binWtr.Write(byteBuf);
Sign up to request clarification or add additional context in comments.

10 Comments

Or you can go unsafe, and safe a precious few cycles :D
Whilst technically correct it might be useful to explain WHY it is faster. Since if the OP needed to write 4GB worth of bytes, this certainly would not work.
@leppie I hope anyone doing that can supply the profiling results showing that it helped enough to justify enabling unsafe code in the project - because I bet it won't do much, not in this case :)
@romkyns My point is that it would be nice to give a deeper understanding of the problem at hand, WHY this is faster, and how to apply the lessons learnt.
@Aron and even if it were 4gb, you'd simply divide it into the biggest chunks that do work
|
1

You're writing the bytes 1 by 1, of course it's going to be slow.

You could do the writing in memory to an array and then write the array to disk all at once like this :

var arr = new double[1000000];
using(var strm = new MemoryStream())
using (var bw = new BinaryWriter(strm)) 
{ 
    foreach(var d in arr)
    {
        bw.Write(d);
    }
    bw.Flush();
    File.WriteAllBytes("myfile.bytes",strm.ToArray());
}

2 Comments

This might not be deserving of two downvotes anymore, but this isn't a great way of speeding up this particular task due to the unnecessary use of extra memory, so no upvote either...
@romkyns well aint that many ways to speed up something without knowing the issue but usually when something involve disk access and is slow, the best way to speed it up is to process it all at once, and the only way to process it all at once, is to have it all at once ready to be processed, which means memory allocated just for that, most of the time speed vs memory is a tradeof, can't get both

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.