6

I am trying to understand how JVM internally implements an array of primitive type, like int []

My question is in two parts:

1 - Does it use a ByteBuffer internally? Where can I find the source code and then perhaps change it according to my needs (to make a modified JVM of my own).

2 - Is there any way to trick javac to not use the build-in implementation of int [] but rather use an implementation provided by a library in lets say classpath -cp? Is this possible and how?

My motivation is to declare this int [] in a memory outside of JVM (using allocateDirect()) and access it outside from a native JNI code. This should avoid the memory copy overhead.

-B

Improve this question

2 Answers 2

Reset to default
1

Instead of using a Java IntBuffer or JNI, you can use sun.misc.Unsafe to allocate and access raw shared memory. This is Dangerous, but it is the absolute fastest way to access shared memory from Java and another process.

A helpful guide to the Unsafe methods is here.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

The sun.misc.Unsafe surely gives the option of allocating native memory but the issue persist as the user arrays (ordinary, normal, primitive arrays) are not implicitly allocated natively.
Right. It's an alternative to the Java arrays that gives easier interop with other processes. Although I wouldn't be surprised if you could do some wildly unsafe trickery to convince the JVM that it's accessible as a int[] array with the standard syntax - the in-memory header for an int[] is pretty simple. Hmm...
do you mean to have a custom JVM?
A custom JVM is not necessary; one can use Unsafe to do arbitrary memory mutations. Some of those mutations let you create a structure with the same layout, off-heap, as a Java array. The only question is whether you can then take that structure and cast it to a "proper" int[]. I somewhat believe this is possible but I haven't tried it.
0

  1. int[] is the primitive type. ByteBuffer may be based on an int[] but the opposite is certainly not true. It will almost cerainly be a word-aligned contiguous block of memory with 32 bit values stored in each word of the block. Several JVM's are open source, you can go look at the code if you really want to but it will be advanced stuff.

  2. There will be no easy way to do this.

It sounds like you are trying to do something in a very complicated way. It may be better to describe the actual problem you are trying to solve rather than asking about problems with an attempted solution to that problem.

Improve this answer

6 Comments

I am using MPJ Express (MPI implementation in Java) it uses an internal buffering layer to buffer the user data. The issue is with this buffering. On high-end interconnects (Infiniband) this copying of data into the internal buffering layer significantly decreases the performance, i.e relatively low bandwidth as compared to native C MPI implementations (like MVAPICH or Open MPI). So I want to bypass this buffering layer and directly access the user data without changing API (the user programs).
@Bibrak do you use System.arraycopy to copy data?
@AlexeiKaigorodov but doesnt it do a copy? I want to do a Zero-Copy
@Bibrak System.arraycopy is much faster than for(){} loop. First try it, and if it is still bottleneck, search for another approaches.
@AlexeiKaigorodov actually MPJ Express uses ByteBuffer as internal storage before transferring, asIntBuffer is to retrieved IntBuffer from the ByteBuffer and data is packed into it from the user int [] as the src, using the standard API functions of the IntBuffer put(int[] src, int offset, int length). The optimization is there. But I just was to remove this internal buffering and directly access the user int [] in JNI.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.