elasticsearch out of memory error

Question

We are using elasticsearch 0.90.0 and java version "1.7.0_25". We migrate data from oracle DB to Hadoop through a executable jar kept at DB server.after 15-20 mins of successful running, we get following exception

Caused by: java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:597)
    at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:95)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:51)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:99)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:69)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
    at org.elasticsearch.transport.netty.NettyTransport.doStart(NettyTransport.java:240)
    at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at org.elasticsearch.transport.TransportService.doStart(TransportService.java:90)
    at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:179)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:119)

No exception is caught either in namenode/datanode or elasticsearch logs. This error is caught at DB Server but I think it is related to elastic search.

How many indices do you have? How much memory is allocated to Elastic Search? Is this a recent occurrence (assume it is since you are running an old version of ES). You may have crossed a point where you either need to delete old indices or increase the memory allocated to ES. — Alcanzar
– Alcanzar, Commented Aug 7, 2014 at 14:08

Community · Accepted Answer · 2017-05-23 12:19:23Z

0

Looks like you're creating too many Netty clients is my guess which is in turn eating up all your threads. Perhaps wrap your Netty client pool with a service that inject? See Helter Skelter's comment on this answer https://stackoverflow.com/a/5253186/266531

edited May 23, 2017 at 12:19

CommunityBot

11 silver badge

answered Jan 7, 2015 at 22:28

Kirby

16k11 gold badges97 silver badges110 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

elasticsearch out of memory error

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related