How to write a sequence of bytes from a file to a byte array without padding the array with null bytes?

Question

I have

[13,132,32,75,22,61,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

I want

[13,132,32,75,22,61,50]

I have an array of bytes size 1048576 that I have written to using a file stream. Starting at a particular index in this array until the end of the array are all null bytes. There might be 100000 bytes with values and 948576 null bytes at the end of the array. When I don't know the size of a file how do I efficiently create a new array of size 100000 (i.e. same as total bytes in unknown file) and write all bytes from that file to the byte array?

byte[] buffer = new byte[0x100000];
int numRead = await fileStream.ReadAsync(buffer, 0, buffer.length); // byte array is padded with null bytes at the end

There's no way you can avoid null-bytes in the buffer when reaching the EOF unless you randomly manage to size your buffer perfectly. That's expected when you read a file like this. The buffer - as the definition states - is used to temporarily store data while it is being moved from one place to another, so why do you want to avoid null-bytes in the buffer? — Xerillio
– Xerillio, Commented Apr 26, 2020 at 12:08
Is the problem that you have an array with null bytes and you don't want to write them to a file? Or is the problem that you have a file padded with null bytes and you don't want to read them, or you want to discard them after reading? — Patrick Tucci
– Patrick Tucci, Commented Apr 26, 2020 at 12:27
@Xerillio I want to avoid null bytes because after decoding the sequence of bytes in the array to a string, thousands of null bytes are appended to the end of the string. Normally this is not a problem but let's say I need to split the string into an array of strings at every instance of the new line character '\n'then the null bytes end up occupying the last element in the array, and any subsequent operations on the array could be affected by the empty row of null bytes. I know I could remove the null bytes element after the fact but I wanted to know if there was a preventative measure. — bit
– bit, Commented Apr 26, 2020 at 14:09
@PatrickTucci I hope my above comment also answered your question. — bit
– bit, Commented Apr 26, 2020 at 14:11

Xerillio · Accepted Answer · 2020-04-26 14:26:32Z

You're stating in the comments that you're just decoding the byte array into a string, so why not read the file contents as a string, such as:

var contents = File.ReadAllText(filePath, Encoding.UTF8);
// contents holds all the text in the file at filePath and no more

or if you want to use a stream:

using (var sr = new StreamReader(path)) 
{
    // Read one character at a time:
    var c = sr.Read();

    // Read one line at a time:
    var line = sr.ReadLine();

    // Read the whole file
    var contents = sr.ReadToEnd();
}

If you, however, insist on going through a buffer you cannot avoid part of the buffer being empty (having null-bytes) when you reach the end of the file but that's where the return value of ReadAsync saves the day:

byte[] buffer = new byte[0x100000];
int numRead = await fileStream.ReadAsync(buffer, 0, buffer.length);

var sectionToDecode = new byte[numRead];
Array.Copy(buffer, 0, sectionToDecode, 0, numRead);
// Now sectionToDecode has all the bytes that were actually read from the file

Collectives™ on Stack Overflow

How to write a sequence of bytes from a file to a byte array without padding the array with null bytes?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related