Sending big file using FileInputStream/ObjectOutputStream

Question

I need help on my homework, any help will be much appreciated. I can send small files without a problem. But when i try to send let’s say a 1GB file byte array sends OutOfMemoryError so i need a better solution to send file from server to client. How can i improve this code and send big files, please help me.

Server Code:

    FileInputStream fis = new FileInputStream(file);
    byte[] fileByte = new byte[fis.available()]; //This causes the problem.
    bytesRead = fis.read(fileByte);
    oos = new ObjectOutputStream(sock.getOutputStream());
    oos.writeObject(fileByte);

Client Code:

    ois = new ObjectInputStream(sock.getInputStream());
    byte[] file = (byte[]) ois.readObject();
    fos = new FileOutputStream(file);
    fos.write(file);

"sends out of memory exception" No such thing in Java, DYM OutOfMemoryError? If so, be specific, if not please explain (also specifically). — Andrew Thompson
– Andrew Thompson, Commented May 30, 2012 at 15:38
BTW - your first code seems to imply the input file always contains serialized objects. Is that the case here? — Andrew Thompson
– Andrew Thompson, Commented May 30, 2012 at 15:44
Oh, im sorry it is OutOfMemoryError not exception. And i don't know what you mean by if the file contains serialezed objects, server sends whatever file the client requests. — Ozzy
– Ozzy, Commented May 30, 2012 at 16:20
ObjectOutputStream and ObjectInputStream are used for serialized objects. If the content is 'any old bytes' the code should not use them. — Andrew Thompson
– Andrew Thompson, Commented May 30, 2012 at 16:22

Erik-RW · Accepted Answer · 2012-05-30 15:40:52Z

4

Don't read the whole file into memory, use a small buffer and write while you are reading the file:

BufferedOutputStream bos = new BufferedOutputStream(sock.getOutputStream())

File file = new File("asd");
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
byte[] buffer = new byte[1024*1024*10];
int n = -1;
while((n = bis.read(buffer))!=-1) {
  bos.write(buffer,0,n):
}

Use Buffered* to optimize the writing and reading from Streams

answered May 30, 2012 at 15:40

Erik-RW

1734 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Andrew Thompson · Accepted Answer · 2012-05-30 15:46:37Z

4

Just split the array to smaller chunks so that you don't need to allocate any big array.

For example you could split the array into 16Kb chunks, eg new byte[16384] and send them one by one. On the receiving side you would have to wait until a chunk can be fully read and then store them somewhere and start with next chunk.

But if you are not able to allocate a whole array of the size you need on server side you won't be able to store all the data that you are going to receive anyway.

You could also compress the data before sending it to save bandwidth (and time), take a look at ZipOutputStream and ZipInputStream.

edited May 30, 2012 at 15:46

Andrew Thompson

169k42 gold badges224 silver badges441 bronze badges

answered May 30, 2012 at 15:38

Jack

134k33 gold badges252 silver badges349 bronze badges

Comments

dandan78 · Accepted Answer · 2012-05-31 08:36:19Z

1

Here's how I solved it:

Client Code:

 bis=new BufferedInputStream(sock.getInputStream());
 fos = new FileOutputStream(file);
 int n;
 byte[] buffer = new byte[8192];
 while ((n = bis.read(buffer)) > 0){
 fos.write(buffer, 0, n);}

Server Code:

 bos= new BufferedOutputStream(sock.getOutputStream());
 FileInputStream fis = new FileInputStream(file);
 BufferedInputStream bis = new BufferedInputStream(fis);
 int n=-1;
 byte[] buffer = new byte[8192];
 while((n = bis.read(buffer))>-1) 
 bos.write(buffer,0,n);

edited May 31, 2012 at 8:36

dandan78

14.1k13 gold badges68 silver badges82 bronze badges

answered May 31, 2012 at 0:04

Ozzy

211 gold badge1 silver badge5 bronze badges

1 Comment

Richard Chase Over a year ago

I followed this example and in the while loop on the receiving end, I print "Writing file...". What I see is the receiver end up in an infinite loop and never completes the receive. I can see the file that it received and the file size eventually gets to how big it should be and then it stops but the client never stops thinking its receiving data. If I close the receiver and try to open the file, it's corrupt.

jtahlborn · Accepted Answer · 2012-05-30 15:48:45Z

Depending on whether or not you have to write the code yourself, there are existing libraries which solve this problem, e.g. rmiio. If you are not using RMI, just plain java serialization, you can use the DirectRemoteInputStream, which is kind of like a Serializable InputStream. (this library also has support for things like auto-magically compressing the data).

Actually, if you are only sending file data, you would be better off ditching the Object streams and use DataInput/DataOutput streams. first write an integer indicating the file length, then copy the bytes directly to the stream. on the receiving side, read the integer file length, then read exactly that many bytes.

when you copy the data between streams, use a small, fixed size byte[] to move chunks of data between the input and output streams in a loop. there are numerous examples of how to do this correctly available online (e.g. @ErikFWinter's answer).

Collectives™ on Stack Overflow

Sending big file using FileInputStream/ObjectOutputStream

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related