Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=1)

Question

I have started metastore and hiveserver2

#./hive --service metastore
#./hive --service hiveserver2

When I excute below query

#./beeline -u jdbc:hive2://192.168.0.10:10000 -e 'select count(*) from test_tb'  --hiveconf hive.root.logger=DEBUG,console --verbose=true

It throws below error

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=1)
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
        at org.apache.hive.beeline.Commands.execute(Commands.java:736)
        at org.apache.hive.beeline.Commands.sql(Commands.java:657)
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:804)
        at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:608)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:630)
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:368)
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:351)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Beeline version 0.13.1 by Apache Hive

hiveserver2 log below

6/06/14 10:57:32 [main]: WARN common.LogUtils: DEPRECATED: Ignoring hive-default.xml found on the CLASSPATH at /data/offline/apache-hive-0.13.1-bin/conf/hive-default.xml
16/06/14 10:57:32 [main]: INFO metastore.HiveMetaStore: Starting hive metastore on port 9083
16/06/14 10:57:32 [main]: INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/06/14 10:57:32 [main]: INFO metastore.ObjectStore: ObjectStore, initialize called
16/06/14 10:57:33 [main]: INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
16/06/14 10:57:33 [main]: INFO metastore.ObjectStore: Initialized ObjectStore
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Added admin role in metastore
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Added public role in metastore
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Starting DB backed MetaStore Server
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Started the new metaserver on port [9083]...
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Options.minWorkerThreads = 200
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: Options.maxWorkerThreads = 100000
16/06/14 10:57:34 [main]: INFO metastore.HiveMetaStore: TCP keepalive = true
16/06/14 10:57:40 [pool-3-thread-1]: INFO metastore.HiveMetaStore: 1: source:/10.234.177.127 get_table : db=default tbl=test_tb
16/06/14 10:57:40 [pool-3-thread-1]: INFO HiveMetaStore.audit: ugi=qspace       ip=/10.234.177.127      cmd=source:/192.168.0.10 get_table : db=default tbl=test_tb
16/06/14 10:57:40 [pool-3-thread-1]: INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/06/14 10:57:40 [pool-3-thread-1]: INFO metastore.ObjectStore: ObjectStore, initialize called
16/06/14 10:57:40 [pool-3-thread-1]: INFO metastore.ObjectStore: Initialized ObjectStore

That would be great. It says execution failed for statement, but your statement looks fine — OneCricketeer
– OneCricketeer, Commented Jun 14, 2016 at 3:30
@cricket_007 Thanks, the link is helpless--- stackoverflow.com/a/19097467/2308683 — lucasys feng
– lucasys feng, Commented Jun 22, 2016 at 12:08

apublius · Accepted Answer · 2016-06-16 00:03:16Z

1

I see you are using Hiveserver2. For such aggregations, depending upon configuration, I believe you may need to set the number of reducers prior to execution. With Hive you can use this syntax:

SET mapred.reduce.tasks=1

However on Hive2 I have noticed I need to use:

SET mapreduce.job.reduces=1

I hope this helps! Previously I had a the same error message and changing this resolved the problem for me.

answered Jun 16, 2016 at 0:03

apublius

1043 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

lucasys feng Over a year ago

I tried, but it still throw the same error, still thanks

Community · Accepted Answer · 2020-06-20 09:12:55Z

1

This could be a permission issue,Try starting beeline as follows

beeline

!connect jdbc:hive2://hadoop7:10000/default

enter your user and password

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Sep 16, 2018 at 13:15

wise_w

294 bronze badges

1 Comment

Varun Rajan Over a year ago

was a permission issue for me! specifically i needed to use small letters in my user id as opposed to the actual caps in it. like remigiusz mentioned below it apparently works fine for select * queries, but fails for everything else

remigiusz boguszewicz · Accepted Answer · 2019-02-25 15:46:43Z

Confirming that in my case it was caused by permission issue (as pointed already by wise_w) what was weird is that it looked like plain select * worked fine but anything more elaborate did not.

I tried the following:

$ beeline -u jdbc:hive2://server1:10000
(connected fine, no errors)
> select * from table1;
-> worked fine

> select count(*) from table1;
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=1)
or
ERROR : Job Submission failed with exception 'org.apache.hadoop.security.AccessControlException(Permission denied: user=anonymous, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

The solution started with wise_w suggestion

$ beeline --incremental=true
$ !connect jdbc:hive2://server1:10000/default
Enter username for jdbc:hive2://server1:10000/default: hive
Enter password for jdbc:hive2://server1:10000/default: **** (this is hive as well in my test setup)

-> and now everything worked fine.

Now one liner

$ beeline -u jdbc:hive2://server1:10000 -n hive -p hive --incremental=true

I thing the confision came from the fact, that connecting like:

$ beeline -u jdbc:hive2://server1:10000

does not throw any error.

swapnil shashank · Accepted Answer · 2019-11-14 06:56:35Z

0

Setting the below parameters, it is important in this case:

set mapred.job.queue.name=<your queue name>;
set mapreduce.job.reduces=1;

answered Nov 14, 2019 at 6:56

swapnil shashank

1,0179 silver badges11 bronze badges

Comments

juhee · Accepted Answer · 2017-06-16 07:20:35Z

-1

Add the below snippet before running you your query:

SET hive.auto.convert.join=false;
SET mapreduce.map.memory.mb = 16384; 
SET mapreduce.map.java.opts='-Djava.net.preferIPv4Stack=true -Xmx13107M';
SET mapreduce.reduce.memory.mb = 13107; 
SET mapreduce.reduce.java.opts='-Djava.net.preferIPv4Stack=true -Xmx16384M';
set hive.support.concurrency = false;

answered Jun 16, 2017 at 7:20

juhee

1

Collectives™ on Stack Overflow

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=1)

5 Answers 5

1 Comment

1 Comment

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related