Writing large files using BufferedOutputStream

Question

I am trying read and write large files (larger than 100 MBs) using BufferedInputStream & BufferedOutputStream. I am getting Heap Memory issue & OOM exception.
The code looks like :

BufferedInputStream buffIn = new BufferedInputStream(iStream);
/** iStream is the InputStream object  **/

BufferedOutputStream buffOut=new BufferedOutputStream(new FileOutputStream(file));
byte []arr = new byte [1024 * 1024];
int available  = -1;
while((available = buffIn.read(arr)) > 0) {   
    buffOut.write(arr, 0, available); 
}      
buffOut.flush();
buffOut.close();

My question is when we use the BufferedOutputStreeam is it holding the memory till the full file is written out ?
What is the best way to write large files using BufferedOutputStream?

it's buffering until you flush it. flush it in the while loop. — jcomeau_ictx
– jcomeau_ictx, Commented Feb 24, 2012 at 4:24
@jcomeau_ictx - no, BufferedOutputStream does not buffer until you flush it, it has a fixed internal buffer size. — jtahlborn
– jtahlborn, Commented Feb 24, 2012 at 4:46
@HunterMcMillen Flushing inside the loop completely negates the point of using a BufferedOutputStream at all, and it won't do anything to the memory usage whatsoever. — user207421
– user207421, Commented Feb 24, 2012 at 9:15
possible duplicate of What is an OutOfMemoryError and how do I debug and fix it — Raedwald
– Raedwald, Commented Sep 23, 2014 at 12:37

jtahlborn · Accepted Answer · 2012-02-24 04:40:25Z

5

there is nothing wrong with the code you have provided. your memory issues must lie elsewhere. the buffered streams have a fixed memory usage limit.

the easiest way to determine what has caused an OOME, of course, is to have the OOME generate a heap dump and then examine that heap dump in a memory profiler.

answered Feb 24, 2012 at 4:40

jtahlborn

53.8k5 gold badges80 silver badges122 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

BillRobertson42 Over a year ago

This is correct. But, there is a problem with the code, buffOut.write() might write fewer bytes than what you ask it to write, in which case, you would need to continue writing until you've written all of the available bytes.

jtahlborn Over a year ago

@Bill - no, that is not true. write() always writes all the bytes (it has no return value).

Paul Jowett Over a year ago

Are you running this code in many threads at once? It's pretty hard to see that this is the source of your out of memory error.

Collectives™ on Stack Overflow

Writing large files using BufferedOutputStream

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related