ASP.net memory usage during download

Question

On an ASP.net site at my place of work, the following chunk of code is responsible for handling file downloads (NOTE: Response.TransmitFile is not used here because the contents of the download are being streamed from a zip file):

private void DownloadFile( Stream stream)
{
        int bytesRead;
        int chunkSize = 1048576; //1MB

        byte[] readBuffer = new byte[chunkSize];
        while ((bytesRead = stream.Read(readBuffer, 0, readBuffer.Length)) != 0)
            {
                if(!Response.IsClientConnected)
                    break;
                byte[] chunk = new byte[bytesRead];
                Array.Copy(readBuffer,0,chunk,0,bytesRead);
                Response.BinaryWrite(chunk);
                Response.Flush();
        }
        stream.Close();
}

Our users frequently download multi-hundred MB files, which can chew up server memory pretty fast. My assumption is that this is due to response buffering. Does that make sense?

I've just read about the 'buffer' property of the Response object. If I set that to false, will that prevent the Response.BinaryWrite() calls from buffering the data in memory? In general, what is a good way to limit memory usage in this situation? Perhaps I should stream from the zip to a temporary file, then call Response.TransmitFile()?

EDIT: In addition to possible solutions, I'm very interested in explanations of the memory usage issue present in the code above. Why would this consume far more than 1MB, even though Response.Flush is called on every loop iteration? Is it just the unnecessary heap allocation that occurs on every loop iteration (and doesn't get GC'd right away), or is there something else at work?

Ray · Accepted Answer · 2010-01-22 21:33:09Z

5

Here is some code that I am working on for this. It uses an 8000 byte buffer to send the file in chunks. Some informal testing on a large file showed a significant decrease in memory allocated.

int BufferSize = 8000;
FileStream stream = new FileStream(fileName, FileMode.Open, FileAccess.Read);
try {
  long fileSize = stream.Length;

  long dataLeftToRead = fileSize;
  int chunkLength;
  buffer = new Byte[BufferSize];

  while (dataLeftToRead > 0) {
    if (!Response.IsClientConnected) {
      break;
    }
    chunkLength = stream.Read(buffer, 0, BufferSize);

    Response.OutputStream.Write(buffer, 0, chunkLength);
    Response.Flush();

    dataLeftToRead -= chunkLength;
  }
}
finally {
  if (stream != null) {
    stream.Close();
}

edited to fix a syntax error and a missing value

edited Jan 22, 2010 at 21:33

answered Jan 22, 2010 at 21:25

Ray

21.9k6 gold badges51 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Rubens Farias Over a year ago

pedant comment: 8k = 8192 bytes;

Ray Over a year ago

thanx - fixed - we geeks should be scrupulously accurate

Odrade Over a year ago

What is the difference between using Response.Write vs. writing directly to the Response.OutputStream object?

Ray Over a year ago

Response.Write writes out string or characters. Using the stream allows you to send binary data.

Odrade Over a year ago

So, where does the memory savings come from, other than using a smaller read buffer (8000 bytes vs. 1MB)? Is it just that I'm allocating memory on the heap with the needless call to 'new' on each loop iteration, and garbage collection may not happen for a while?

|

Collectives™ on Stack Overflow

ASP.net memory usage during download

1 Answer 1

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related