Java: How to Create Large 2-Dimensional Array

Question

I'm trying to write a simple program that will simulate a scenario described in a probability question. In order to do so, I tried to create and populate a large 2-dimensional array, but the compiler continues to say that the java heap size isn't large enough and when I allocate more memory to the JVM, the process eats up my CPU to the point where my laptop crashes (mind you, it's an ultrabook with an i7 and 8GB of ram). Is this just not possible/recommended in Java? Is there another way? Please help!

This is the line that's the problem (2-dimensional array with 5^12 rows and 12 columns):

int[][] sequences = new int[244140625][12];

P.S. I'm a bit of a beginner when it comes to programming... thank you in advance!

So a bit of advice: if you want a data structure this big, you want a database, not an in-memory array. — Makoto
– Makoto, Commented Oct 8, 2015 at 0:57
I see, but how would I implement a database? Is it just another data structure like Arrays or ArrayLists? — DavidSGK
– DavidSGK, Commented Oct 8, 2015 at 1:01
It's backed by a data structure - often a B-tree - but you'd usually look at DBMS solutions like MySQL or PostgreSQL. — Makoto
– Makoto, Commented Oct 8, 2015 at 1:02
So I guess that means this simulation won't be possible with just Java on my system at least... bummer. I'll try looking into databases but I don't think I'll get very far. :( — DavidSGK
– DavidSGK, Commented Oct 8, 2015 at 1:07
I don't know what you're trying to simulate but maybe writing data to disk manually can solve the problem for you. Like with 12 files instead of a 2d array. Also H2 is an embeddable pure java database and you don't need an extra database server. — zapl
– zapl, Commented Oct 8, 2015 at 1:20

zapl · Accepted Answer · 2015-10-08 01:02:55Z

4

each int in Java is 4 byte large. 244140625 of them take already 931MiB of RAM. You want this x 12. So 10.91GiB. Java could to my knowledge handle that but those 8 gig of RAM you have are not enough.

A database like H2 (http://www.h2database.com/html/main.html) is easy to integrate and can swap to disk. Or you come up with a better algorithm that doesn't need to store 5^12x12 values.

answered Oct 8, 2015 at 1:02

zapl

64.1k10 gold badges126 silver badges160 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Andreas · Accepted Answer · 2015-10-08 01:10:48Z

2

It's actually worse than they say.

An int[12] is (at minimum) 4 bytes for class pointer + 4 bytes for array length + 12 * 4 bytes for values = 56 bytes.

The outer array then uses 4 bytes per pointer, so you get 244,140,625 * (56 + 4) = 14,648,437,500 = 13.6 Gb of memory.

It might work if you run 64-bit Java with -Xmx16g, but it'll run forever with all the paging going on when you only have 8 Gb, and that assumes that pointers are only 4 bytes (compressed).

answered Oct 8, 2015 at 1:10

Andreas

160k13 gold badges164 silver badges262 bronze badges

Comments

ashiquzzaman33 · Accepted Answer · 2015-10-08 01:09:47Z

0

Your array size

 244140625*12*4 = 11718750000 byte
 11718750000%(1024*1024*2024) = 10.91 GB

So, only your array need 10.91 GB memory, so it is not feasible to implement it with array. Alternatively you can use database like MySQL, PostgreSQL or other DBMS solutions those are built for handle huge amount of data.

edited Oct 8, 2015 at 1:09

answered Oct 8, 2015 at 1:04

ashiquzzaman33

5,7375 gold badges34 silver badges45 bronze badges

Comments

Hamdy Abd El Fattah · Accepted Answer · 2021-04-06 11:36:02Z

0

Tru this

ArrayList<Integer>[] arr= (ArrayList<Integer>[])new ArrayList[n];

Then

for (int i = 0; i < n; i++) {
    arr[i] = new ArrayList<Integer>();
}

Then

    arr[0].add(5);

answered Apr 6, 2021 at 11:36

Hamdy Abd El Fattah

1,8671 gold badge19 silver badges20 bronze badges

Collectives™ on Stack Overflow

Java: How to Create Large 2-Dimensional Array

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related