Hadoop JAR command - Setting java.library.path

Question

I'm trying to run a java program on Hadoop cluster. Here's the command-

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/*:/home/rgupta/bdAnalytics/lib/*
hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub >  $logsFldr/subsHdpOMQ_$1.log 2>&1 &
#java -Djava.library.path=/usr/local/lib -classpath  class/:lib/:lib/jzmq-2.1.3.jar  bigdat.twitter.queue.TweetOMQSub > log/subsFilterOMQ_$1.log 2>&1 &

This throws following error -

Exception in thread "main" java.lang.UnsatisfiedLinkError: no jzmq in java.library.path

If I use the Java native command above, it works fine. Also, the hadoop node where I m trying to test it, does have the necessary jzmq jars under /usr/local/lib directory. Is there a way I can set java.library.path to Hadoop JAR command. Please suggest how can I fix this.

Engineiro · Accepted Answer · 2013-07-01 19:20:17Z

1

sorry misread your question so editing:

you should be able to use the libjars option

In your case:

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/:/home/rgupta/bdAnalytics/lib/

hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub -libjars /usr/local/lib ...

edited Jul 1, 2013 at 19:20

answered Jul 1, 2013 at 18:35

Engineiro

1,1467 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user2508012 Over a year ago

Thanks! I just tried this out, but no luck! I still get the same exception

SSaikia_JtheRocker · Accepted Answer · 2013-07-04 03:39:29Z

1

Try export HADOOP_OPTS=$HADOOP_OPTS -Djava.library.path=/usr/local/lib

and export other jars the usual way you are doing before running a job - using HADOOP_CLASSPATH

Hope this helps.

edited Jul 4, 2013 at 3:39

answered Jul 1, 2013 at 20:34

SSaikia_JtheRocker

5,0531 gold badge25 silver badges43 bronze badges

1 Comment

SSaikia_JtheRocker Over a year ago

Hey check it now, I had a minor typo, I missed $ before HADOOP_OPTS

Collectives™ on Stack Overflow

Hadoop JAR command - Setting java.library.path

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related