Cassandra : java.lang.OutOfMemoryError: Java heap space

Question

I am using cassandra 2.0.8 and getting this exception

INFO 16:44:50,132 Initializing system.batchlog
 INFO 16:44:50,138 Initializing system.sstable_activity
 INFO 16:44:50,142 Opening /var/lib/cassandra/data/system/sstable_activity/system-sstable_activity-jb-10 (826 bytes)
 INFO 16:44:50,142 Opening /var/lib/cassandra/data/system/sstable_activity/system-sstable_activity-jb-9 (827 bytes)
 INFO 16:44:50,142 Opening /var/lib/cassandra/data/system/sstable_activity/system-sstable_activity-jb-11 (825 bytes)
 INFO 16:44:50,150 reading saved cache /var/lib/cassandra/saved_caches/system-sstable_activity-KeyCache-b.db
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid3460.hprof ...
Heap dump file created [13378724 bytes in 0.071 secs]
ERROR 16:44:50,333 Exception encountered during startup
java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.<init>(ArrayList.java:144)
    at org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:120)
    at org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:365)
    at org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:119)
    at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:262)
    at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:421)
    at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:392)
    at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:309)
    at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:266)
    at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
    at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
    at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:536)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:261)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.<init>(ArrayList.java:144)
    at org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:120)
    at org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:365)
    at org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:119)
    at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:262)
    at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:421)
    at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:392)
    at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:309)
    at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:266)
    at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
    at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
    at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:536)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:261)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
Exception encountered during startup: Java heap space

Can anyone tell me the reason and solution:

Aaron · Accepted Answer · 2014-09-04 16:03:59Z

3

Reach out to cassandra/conf/cassandra-env.sh location

Check out the current heap size You can assign at max of 1/2 RAM to the HEAP

#MAX_HEAP_SIZE="4G" 
#HEAP_NEWSIZE="800M"

if you are changing your current heap-size then remove comment..

MAX_HEAP_SIZE="4G"
HEAP_NEWSIZE="800M"

edited Sep 4, 2014 at 16:03

Aaron

58k11 gold badges123 silver badges142 bronze badges

answered Sep 4, 2014 at 11:25

Helping Hand..

2,4204 gold badges34 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Manish Kumar Over a year ago

ok i will check that. I check 'commitlogs' firectory and saw that around 250MB total size of files are there ,is that be also a reason?

Manish Kumar Over a year ago

I tried with MAX_HEAP_SIZE="600MB" HEAP_NEWSIZE="300MB" but got

Invalid initial heap size: -Xms600MB Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.

Helping Hand.. Over a year ago

your machine has 1GB RAM? you can assign 1/2 of RAM of your machine to MAX_HEAP_SIZE, and it has to be HEAP_NEWSIZE="600M" (only M not MB)..

Manish Kumar Over a year ago

its 4gb ram. does the size of commit log file is the reason?

Helping Hand.. Over a year ago

if you are using linux then execute the free command before doing any process it will show the available memory-size, so you can get some idea.

|

Chris Lohfink · Accepted Answer · 2014-09-05 16:48:07Z

3

Its possible your key cache is taking up too much space (since thats where it died) but it seems unlikely. You can try to delete your KeyCache before starting

/var/lib/cassandra/saved_caches

and set

key_cache_size_in_mb: 0

in your cassandra.yaml as a test (I would not recommend this permanently) to have it disabled.

You can actually determine whats filling up your heap by opening up the java_pid3460.hprof file it created in yourkit or some heap analyzer to determine whats taking up the space. There may be something funny going on, very strange to be dying at 13mb or so (size of heap).

answered Sep 5, 2014 at 16:48

Chris Lohfink

16.5k1 gold badge32 silver badges39 bronze badges

Comments

András Váczi · Accepted Answer · 2016-06-22 10:24:31Z

-3

Delete all log files in usr/local/var/lib/cassandra/commitlog/ and restart Cassandra.

edited Jun 22, 2016 at 10:24

András Váczi

3,02225 silver badges30 bronze badges

answered Nov 23, 2015 at 19:54

Abiola

4066 silver badges6 bronze badges

1 Comment

András Váczi Over a year ago

How would this solve the problem? Also, wouldn't it just crash Cassandra under circumstances?

Collectives™ on Stack Overflow

Cassandra : java.lang.OutOfMemoryError: Java heap space

3 Answers 3

7 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related