OutofMemory error for spark streaming job

Question

I have a spark streaming job running in Hortonworks cluster . I am running it in cluster mode through yarn , the job is shown as running in UI , but it is having the below exception in driver logs

Exception in thread "JobGenerator" java.lang.OutOfMemoryError: Java heap space

dileepVikram · Accepted Answer · 2018-10-12 09:36:10Z

1

I fixed the issue by specifying driver-memory in spark-submit command.because the memory issue was in driver

answered Oct 12, 2018 at 9:36

dileepVikram

9324 gold badges16 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

OutofMemory error for spark streaming job

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related