Cassandra generates “java.lang.OutOfMemoryError: Java heap space” error on startup

Question

I upgraded my system from 13.04 to 13.10 and I could not start Cassandra anymore, getting an out of memory error, so the heap was getting full even though all I was doing at that point was start the server.

I went in and increased the maximum heap size with -Xmx1992m (the default) to -Xms2500m.

I am wondering why would it be required to do such a thing? Could it be that the memory decreased a little bit between running Linux kernel 3.5.0 and moving to kernel 3.11.0 and could that impact Cassandra? (i.e. say I "lost" 200M, now it was allocating 1992Mb instead of 2192Mb with the previous kernel, could that be??)

My concern here is that any time I upgrade I could run in a problem like this unless I force the maximum heap size to a fixed number (instead of letting the script determine the Mb to use dynamically.) Plus, I just cannot continue to increase the limit forever. My RAM is limited... (obviously.)

On startup Cassandra shows the following (broken on 3 lines to make it easier to read here):

xss =  -ea -javaagent:bin/../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities
       -XX:ThreadPriorityPolicy=42 -Xms1994M -Xmx2500M -Xmn400M
       -XX:+HeapDumpOnOutOfMemoryError -Xss256k

Update:

Argh! It was still starting OpenVMS because the $JAVA_HOME variable that points to /usr/lib/jvm/default-java (it is defined in /etc/environment):

JAVA_HOME=/usr/lib/jvm/default-java

And somehow it was setup back to OpenVMS in there. Although I changed the /etc/alternatives/java softlink, and thus java -version in my command line was "working fine" (returning what I'd expect) the Cassandra startup script makes use of the $JAVA_HOME variable!

So I checked the list of available VMs with:

ls -l /usr/lib/jvm

lrwxrwxrwx 1 root root   24 Apr 27  2012 default-java -> java-1.7.0-openjdk-amd64
drwxr-xr-x 4 root root 4096 Feb 23 17:54 java-1.5.0-gcj-4.8-amd64
lrwxrwxrwx 1 root root   20 Sep  2  2012 java-1.6.0-openjdk-amd64 -> java-6-openjdk-amd64
lrwxrwxrwx 1 root root   20 Jul  3  2013 java-1.7.0-openjdk-amd64 -> java-7-openjdk-amd64
drwxr-xr-x 5 root root 4096 Oct  7  2012 java-6-openjdk-amd64
drwxr-xr-x 3 root root 4096 Oct  7  2012 java-6-openjdk-common
drwxr-xr-x 5 root root 4096 Sep 21 20:06 java-7-openjdk-amd64
drwxr-xr-x 8 root root 4096 Sep 18 21:18 java-7-oracle

And ran the following two commands to get Oracle to run:

sudo rm /usr/lib/jvm/default-java
sudo ln -s java-7-oracle /usr/lib/jvm/default-java

However, that did not change the fact that I still need to have a large upper limit in terms of memory.

Oh! I wanted to put the info with parameters and such. I am on 64bit. Using Oracle, although it was set on OpenJVM when I rebooted and did not notice until after I tested so it tried several times with OpenJVM. Both version 7. — Alexis Wilke
– Alexis Wilke, Commented Feb 24, 2014 at 23:29
Oh! I changed the alternative to Oracle, but it looks like that's not enough. It makes use of /usr/lib/jvm/default-java/bin/java and it looks like that would be OpenJVM. Arrrgh! — Alexis Wilke
– Alexis Wilke, Commented Feb 24, 2014 at 23:33
So how much memory does a simple Java test program report available? — Thorbjørn Ravn Andersen
– Thorbjørn Ravn Andersen, Commented Feb 24, 2014 at 23:50
@ThorbjørnRavnAndersen, No idea. I don't write Java code. The -Xms and -Xmx define the available heap to the software. I find it really difficult to understand Cassandra, in part because I do not use Java myself and thus a lot of things are really not clear in that respect alone. — Alexis Wilke
– Alexis Wilke, Commented Feb 25, 2014 at 0:10

Tim B · Accepted Answer · 2014-02-24 23:32:55Z

2

Things like the memory usage of applications, libraries, and even core Java are subject to change without notice. You should expect particularly massive changes to be documented but in most cases a few hundred MB here or there will not be.

It's a general truism that in most cases memory usage increases as software develops. As programs get bigger, get more features, etc then they also consume more memory. The only way to see how much memory you will need for a particular piece of software doing a particular job is to test it - and if the software version changes test it again.

The easiest way is just to make sure you allocate enough headroom in the first place that small increases will not be a problem.

answered Feb 24, 2014 at 23:32

Tim B

41.3k16 gold badges88 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alexis Wilke Over a year ago

Yeah... unfortunately at times it also means things stop working, which is a real annoyance. For example, at this point I have no sound at all. It used to work since I got the computer like 4 or 5 years ago. A shame really.

Collectives™ on Stack Overflow

Cassandra generates “java.lang.OutOfMemoryError: Java heap space” error on startup

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related