Using the "using" statement with StreamWriter/Reader inside loop

Question

Here is my situation.

Read one line from a text file
"Process" the line
Write the "processed" line to a new text file
Loop to #1 and repeat until EOF

Here is how I'm doing it:

using (StreamReader srReader = new StreamReader(strInputFile))
{
    // loop until EOF
    while ((strCurrentLine = srReader.ReadLine()) != null)
    {
        // "process" strCurrentLine...

        // write the "processed" strCurrentLine to a new text file
        using (StreamWriter sw = new StreamWriter(strFullOutputPathFileName, false))
        {
            // write strCurrentLine to the new file
            sw.WriteLine("stuff...");
        }
     }
}

My manager tells me that using the using statement like I am inside of a loop will extremely hinder performance. The reason is because the StreamWriter instance will be created as many times as I'm looping. So, if I looped 1,000 times, I'd have 1,000 instances of my StreamWriter object, which could severely hinder performance.

Is this true? Also, is my method above the best way to accomplish this?

Surely the bigger problem than performance is the fact that you're not appending - so your final file will just be the last line of the input file... — Jon Skeet
– Jon Skeet, Commented Mar 27, 2014 at 20:08
I'd have 1,000 instances Well, no. But you'd have created one 1000 times. — tnw
– tnw, Commented Mar 27, 2014 at 20:08
The using statement isn't a performance problem, but calling "new StreamWriter" 1000 times instead of once is. As the other comments point out, though, the fact that you are overwriting your file 999 times is a bigger issue. — dgvid
– dgvid, Commented Mar 27, 2014 at 20:10
See C# I/O - Difference between System.IO.File and StreamWriter/StreamReader. I prefer using File.ReadAllText() and File.WriteAllText() over StreamReader and StreamWriter. — mason
– mason, Commented Mar 27, 2014 at 20:10

D Stanley · Accepted Answer · 2014-03-27 20:15:14Z

My manager tells me that using the using statement like I am inside of a loop will extremely hinder performance. The reason is because the StreamWriter instance will be created as many times as I'm looping. So, if I looped 1,000 times, I'd have 1,000 instances of my StreamWriter object, which could severely hinder performance.

Is this true?

Well, it's true, but not because you're creating instances, but because you're opening and closing a file 1,000 times. You could create 1,000 strings with almost no impact to performance.

Also, is my method above the best way to accomplish this?

To start, move the writer creation outside of the while loop:

using (StreamReader srReader = new StreamReader(strInputFile))
{
    // write the "processed" strCurrentLine to a new text file
    using (StreamWriter sw = new StreamWriter(strFullOutputPathFileName, false))
    {
        // loop until EOF
        while ((strCurrentLine = srReader.ReadLine()) != null)
        {
            // "process" strCurrentLine...

            // write strCurrentLine to the new file
            sw.WriteLine("stuff...");
        }
     }
}

However, you could also read the entire file into memory, process it, and write it out in one operation. The impact will depend on the processing that's done and whether you want partial results if there's an error.

Abbas · Accepted Answer · 2014-03-27 20:10:14Z

4

Change your code to:

using (StreamReader srReader = new StreamReader(strInputFile))
{
    using (StreamWriter sw = new StreamWriter(strFullOutputPathFileName, false))
    {
        while ((strCurrentLine = srReader.ReadLine()) != null)
        {
            sw.WriteLine("stuff...");
        }
    }
}

And also, check Jon's comment about appending.

answered Mar 27, 2014 at 20:10

Abbas

14.5k6 gold badges43 silver badges77 bronze badges

Comments

brz · Accepted Answer · 2014-03-27 20:12:23Z

2

If your file isn't big you can use File.ReadAllLines to get file lines in a string array and do your processing afterwards.

answered Mar 27, 2014 at 20:12

brz

6,0561 gold badge20 silver badges18 bronze badges

Comments

Andrew Backes · Accepted Answer · 2014-03-27 20:10:17Z

1

Yes, you will be creating many StreamWriters by doing it that way (not many instances running simultaneously, however). A simple way to solve this is by creating the StreamWriter before you even enter the while loop.

answered Mar 27, 2014 at 20:10

Andrew Backes

1,9144 gold badges22 silver badges38 bronze badges

Comments

TylerH · Accepted Answer · 2024-09-23 15:40:35Z

0

You should open and close the file as few times as possible. So open it (create streamwriter) before the while loop. In the loop use sw.WriteLine("stuff...") After the loop call sw.Close().

Your manager is wrong in sense that it won't create 1000 instances because each instance will be released at the end of the iteration, but you'll be opening and closing the file which will impact the performance.

edited Sep 23, 2024 at 15:40

TylerH

21.3k84 gold badges84 silver badges122 bronze badges

answered Mar 27, 2014 at 20:09

Damjan Dakic

2651 silver badge19 bronze badges

3 Comments

Jon Skeet Over a year ago

No, it will create 1000 instances. They won't all be "live" at the same time, but it will definitely create 1000 instances...

Kjartan Over a year ago

Also the claim that each instance will be released at the en of the iteration is imprecise; they may be released then, but there is no guarantee as to when the garbage collector will process them. It will probably happan relatively soon, but unless I'm mistaken, the instances could in theory continue to exist in memory for a while even after they are out of scope.

Damjan Dakic Over a year ago

I agree with both of you Jon and Kjartan.

Collectives™ on Stack Overflow

Using the "using" statement with StreamWriter/Reader inside loop

5 Answers 5

Comments

Comments

Comments

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related