Java NIO/ MappedByteBuffer and map, read part of file in chunks

Question

I want to read parts of a large file in a loop. I had to just read the entire file but that didn't work, I was getting an exception that the file is too large. I changed my code to the listing below. The code below only reads in the first chunk. What do I need to change to move to the next chunk.

   final FileInputStream fis = new FileInputStream(f);
    final FileChannel fc = fis.getChannel();
    final long sizeRead = fc.size() < defaultReadBufferSize ? fc.size() : defaultReadBufferSize;
    final MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, sizeRead);        
    while (bb.hasRemaining()) {                        
        final CharBuffer cb = decoder.decode(bb);
        this.search(f, cb);   
        System.out.println("============>" + cb.length());
        System.out.println("============>" + bb.hasRemaining());            
    }        
    fc.close();

You have about 44 questions without an accepted answer. Perhaps you could try to ask questions which are likely to have a reasonable answer or follow up answers so they can be accepted. — Peter Lawrey
– Peter Lawrey, Commented Jul 11, 2011 at 14:16
Can you show me an example question without an accepted answer? All of my questions have answers. — Berlin Brown
– Berlin Brown, Commented Jul 11, 2011 at 14:23
If you go back through your questions stackoverflow.com/users/10522/berlin-brown Questions marked in yellow have an accepted answer, questions in white do not. Pages 3-12 have questions without accepted answers. — Peter Lawrey
– Peter Lawrey, Commented Jul 11, 2011 at 14:25

Peter Lawrey · Accepted Answer · 2011-07-11 14:36:26Z

3

The problem you have is that character encoded data cannot be accessed this way. i.e. you need to know where the boundaries between characters are.

The cost of accessing the file and character decoding it is likely to be far more expensive than how you read it, so I would use a BufferedReader which will be much simpler as well.

e.g. say you want to read from the 1000th byte. You can do that but you won't know if the 1000th byte is part of a multi-byte character or not.

If you can assume all characters are bytes, the whole issue is much simpler and you don't need a CharBuffer, you can access the ByteBuffer directly which would be much faster.

edited Jul 11, 2011 at 14:36

answered Jul 11, 2011 at 14:15

Peter Lawrey

535k83 gold badges770 silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Berlin Brown Over a year ago

Basically, if I wanted to use filechannel.map, how would I read a file without reading the entire file at once? Is that supported with this api?

Peter Lawrey Over a year ago

Yes, it is, but you wouldn't be able to decode bytes to characters safely unless you could work out where a characters start.

Collectives™ on Stack Overflow

Java NIO/ MappedByteBuffer and map, read part of file in chunks

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related