Multidimension array in java inefficiant memory allocation

Question

Simple program allocating 1 multi-dimentional array causes continous GCs (all garbage collectors) if heap is big enough to to create outermost array. It appears that GC is attempted many times. With product build and 6G heap, allocation takes 3 minutes.

With fast debug build, allocation takes more than 20 minutes.

Allocating 1-dimensional array which occupies same memory is fast and does not throw any out of memory exception Why ?

    public class Test {
    public static void main(String[] args) {
            int a1 = 1;
            long maxMemory = Runtime.getRuntime().maxMemory();
            int s = (int) (Math.sqrt(Math.sqrt((double) maxMemory)));
            System.out.println("s: " + s);
            int[][][][] a2;
    try {
        a2 = new int [s][s][s][s];
        a2 [s-1][s-1][s-1][s-1] = a1;
        if ( a2 [s-1][s-1][s-1][s-1] != 1 ) {
            throw new RuntimeException("Error: " + a2 [s-1][s-1][s-1][s-1]);
        }
    } catch (OutOfMemoryError e) {
            System.out.println("Passed.");
    }
    }
}

Lolo · Accepted Answer · 2015-12-29 07:52:39Z

The first problem I see is that you're computing the maximum heap size instead of the maximum available heap size: you need to account for what's already allocated. For example:

Runtime rt = Runtime.getRuntime();
long availableMemory = rt.maxMemory() - (rt.totalMemory() - rt.freeMemory());

The second problem is that you are trying to create as many int values as the maximum heap size in bytes. Since an int is represented with 4 bytes, you're trying to allocate 4 times more memory than the maximum heap size, this should always throw an OOME. Therefore, the size s should be recomputed as:

int s = (int) (Math.sqrt(Math.sqrt((double) availableMemory))) / 4;

Lastly, you're not just allocating int values, but also references to int[], int[][] and int[][][], as per the multi-dimensionality of your array. So the actual maximum array size should be adjusted for the allocation of these references. If I'm not mistaken, this is equivalent to resolving (or finding a good approximation of) this equation:

availableMem >= s * refSize + s^2 *refSize + s^3 * refSize + s^4 * 4

where refSize is the size of the representation of a reference in the JVM, which may depend on the platform (e.g. 4 bytes for a 32 bits JVM, 8 bytes for 64 bits).

Collectives™ on Stack Overflow

Multidimension array in java inefficiant memory allocation

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related