How to read text file from memorystream without missing bytes

Question

I am writing some code to learn new c# async design patterns. So I thought writing a small windows forms program that counts lines and words of text files and display the reading progress.

Avoiding disk swapping, I read files into a MemoryStream and then build a StreamReader to read text by lines and count.

The issue is I can`t update the progressbar right. I read a file but always there are bytes missing, so the progressbar doesn't fill entirely.

Need a hand or a idea to achieve this. Thanks

public async Task Processfile(string fname)
{

  MemoryStream m;
  fname.File2MemoryStream(out m); // custom extension which read file into
                                  // MemoryStream

  int flen = (int)m.Length;       // store File length
  string line = string.Empty;     // used later to read lines from streamreader


  int linelen = 0;                // store current line bytes
  int readed = 0;                 // total bytes read


    progressBar1.Minimum = 0;     // progressbar bound to winforms ui
    progressBar1.Maximum = flen;

    using (StreamReader sr = new StreamReader(m)) // build streamreader from ms
    {

       while ( ! sr.EndOfStream ) // tried ( line = await sr.ReadLineAsync() ) != null
       {

          line = await sr.ReadLineAsync();

            await Task.Run(() =>
            {

              linelen = Encoding.UTF8.GetBytes(line).Length;  // get & update
              readed += linelen;                              // bytes read

                                                         // custom function
              Report(new Tuple<int, int>(flen, readed)); // implements Iprogress
                                                         // to feed progress bar

             });                     
         }
     }

        m.Close();    //  releases MemoryStream
        m = null;            
 }

Using a MemoryStream probably doesn't do any good; both FileStream and StreamReader are intelligently buffered. — SLaks
– SLaks, Commented Jul 24, 2013 at 17:20
@SLaks Yes, those classes are buffered but I read whole archive into a MemoryStream for perfomance. I think reading files in chunks is slower because need more I/O. — ppk
– ppk, Commented Jul 24, 2013 at 17:27
@ppk - unless you're doing any Seeking on the stream, reading the whole file into a MemoryStream is idiotic, if all you want to do is process the file sequentially. The Operating System is already reading the file into memory intelligently. Or to put it another way, don't try to optimize without measuring. At the moment, you just seem to have guessed that putting it into a MemoryStream will (somehow) yield better performance. — Damien_The_Unbeliever
– Damien_The_Unbeliever, Commented Jul 24, 2013 at 17:42

user2584927 · Accepted Answer · 2013-07-24 17:25:11Z

4

The total length being assigned to flen includes the carriage returns of each line. The ReadLineAsync() function returns a string that does not include the carriage return. My guess is that the amount of missing bytes in your progress bar is directly proportional to the amount of carriage returns in the file being read.

answered Jul 24, 2013 at 17:25

user2584927

863 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ppk Over a year ago

Encoding.Getbytes documentation doesn't say anything about not including carriage returns. However I am going to test it.

Damien_The_Unbeliever Over a year ago

@ppk - it's not Encoding.GetBytes that's removing the carriage returns - it's you calling a function (ReadLineAsync) that will never return a carriage return character

Collectives™ on Stack Overflow

How to read text file from memorystream without missing bytes

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related