2

I am trying to use ByteBuffer properly with BigEndian byte order format..

I have couple of fields which I am trying to put together into a single ByteBuffer before storing it in Cassandra database.

That Byte Array which I will be writing into Cassandra is made up of three Byte Arrays as described below-

short employeeId = 32767;
long lastModifiedDate = "1379811105109L";
byte[] attributeValue = os.toByteArray();

Now I need to snappy compress the attributeValue data before storing it in the Cassandra -

employeeId (do not snappy compressed)
lastModifiedDate (do not snappy compressed)
attributeValue  (snappy compressed it)

Now, I will write employeeId , lastModifiedDate and snappy compressed attributeValue together into a single Byte Array and that resulting Byte Array I will write into Cassandra and then I will be having my C++ program which will retrieve that Byte Array data from Cassandra and then deserialize it to extract employeeId , lastModifiedDate and decompress this attributeValue using snappy from it.

So to do this, I am using ByteBuffer with BigEndian byte order format.

I have put up this code together -

public static void main(String[] args) throws Exception {

        String text = "Byte Buffer Test";
        byte[] attributeValue = text.getBytes();

        long lastModifiedDate = 1289811105109L;
        short employeeId = 32767;

        // snappy compressing it and this line gives BufferOverflowException
        byte[] compressed = Snappy.compress(attributeValue);

        int size = 2 + 8 + 4 + attributeValue.length; // short is 2 bytes, long 8 and int 4

        ByteBuffer bbuf = ByteBuffer.allocate(size); 

        bbuf.order(ByteOrder.BIG_ENDIAN);
        bbuf.putShort(employeeId);
        bbuf.putLong(lastModifiedDate);
        bbuf.putInt(attributeValue.length);
        bbuf.put(compressed); // storing the snappy compressed data

        bbuf.rewind();

        // best approach is copy the internal buffer
        byte[] bytesToStore = new byte[size];
        bbuf.get(bytesToStore);

        // write bytesToStore in Cassandra...

        // Now retrieve the Byte Array data from Cassandra and deserialize it...
        byte[] allWrittenBytesTest = bytesToStore;//magicFunctionToRetrieveDataFromCassandra();

        // I am not sure whether the below read code will work fine or not..
        ByteBuffer bb = ByteBuffer.wrap(allWrittenBytesTest);

        bb.order(ByteOrder.BIG_ENDIAN);
        bb.rewind();

        short extractEmployeeId = bb.getShort();
        long extractLastModifiedDate = bb.getLong();
        int extractAttributeValueLength = bb.getInt();
        byte[] extractAttributeValue = new byte[extractAttributeValueLength];

        bb.get(extractAttributeValue); // read attributeValue from the remaining buffer

        System.out.println(extractEmployeeId);
        System.out.println(extractLastModifiedDate);
        System.out.println(new String(Snappy.uncompress(extractAttributeValue)));

}

Somehow the above code is throwing BufferOverflowException -

Exception in thread "main" java.nio.BufferOverflowException
    at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:165)
    at java.nio.ByteBuffer.put(ByteBuffer.java:813)

Why I am snappy compressing the data before storing it in Cassandra because when I retrieve the data from Cassandra from C++ code, then it should be snappy compressed so it will take lot of less space in our C++ Map. And we will decompressed it only when people will make a call to us.

Can anybody take a look and let me know what wrong I am doing here? And how I should read the data back then?

3
  • 1
    You're not properly calculating size. You really couldn't figure that out from the exception? Commented Feb 4, 2014 at 19:49
  • But it is throwing exception while compressing it Snappy.compress(attributeValue);? Commented Feb 4, 2014 at 20:10
  • Unlikely. Do you see Snappy anywhere in the stack trace? Commented Feb 4, 2014 at 23:05

1 Answer 1

1

You should be using the compressed length when allocating the original ByteBuffer.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.