FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hive/serde2/SerDe

Question

I am processing the twitter data to hive external table, but while creating hive external table I get an error. Please look into the my code in below.

Added the jar file to move the hive/lib location.

Added to the jar file hive by using following command
I have add the flowing jars in hive lib directory:

ADD JAR /usr/local/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;

Please find the external hive table:

CREATE EXTERNAL TABLE Mytweets_raw (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweet_count INT,
retweeted_status STRUCT<text:STRING,tuser:STRUCT<screen_name:STRING,name:STRING>>,
entities STRUCT<urls:ARRAY<STRUCT<expanded_url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
hashtags:ARRAY<STRUCT<text:STRING>>>,
text STRING,
tuser STRUCT<screen_name:STRING,name:STRING,friends_count:INT,followers_count:INT,statuses_count:INT,verified:BOOLEAN,utc_offset:INT,time_zone:STRING>,
in_reply_to_screen_name STRING )
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION 'hdfs://localhost:54310/data/tweets_raw';

After this I am getting following error message, please any body can help on this?

More information: My current environment is

Hadoop 2.9.0
Hive 2.3.2

Welcome to SO. I reformatted your question so that the code is displayed properly. When you want to use a code block in an enumeration you have to indent the code by an additional four spaces. — m00am
– m00am, Commented Dec 12, 2017 at 12:48
Your JAR file needs to be on all nodes of the cluster, or you must add from HDFS — OneCricketeer
– OneCricketeer, Commented Dec 12, 2017 at 13:55
this is single node cluster . but i have already added the hive library. — Madhusudhan
– Madhusudhan, Commented Dec 13, 2017 at 6:49

Amit Phaltankar · Accepted Answer · 2018-10-08 07:49:09Z

This worked for me, I just needed to replace the serde format from 'com.cloudera.hive.serde.JSONSerDe' to 'org.openx.data.jsonserde.JsonSerDe', and the error was resolved in my case. Good luck !!

 CREATE EXTERNAL TABLE tweets4 (
    id BIGINT,
    created_at STRING,
    source STRING,
    favorited BOOLEAN,
    retweet_count INT,
    retweeted_status STRUCT<
    text:STRING,
    `user`:STRUCT<screen_name:STRING,name:STRING>>,
    entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
    text STRING,
    `user` STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
    in_reply_to_screen_name STRING
 )
 ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
 LOCATION '/twitter-project/project1';

data_addict · Accepted Answer · 2017-12-13 13:56:38Z

0

Error code 1 signifies permission issue. It will be really difficult to identify the issue (Mostly environment issue). There can be many reasons why we are getting this error. Try below cases and see if you are getting same error.

1) Check you have permissions to create tables(Hive path).

2) Create simple Hive external table(You can confirm you have access for meta store)

3) Create Simple External Hive table by pointing to HDFS location( You can confirm your access permissions for HDFS path)

4) Create Simple External Hive table by pointing to local path ( You can confirm your access permissions for local path)

5) Check you have 777 permissions for the jar location.

I suggest try multiple cases where there is possibility of permission issue apart from the above.

answered Dec 13, 2017 at 13:56

data_addict

8944 gold badges17 silver badges37 bronze badges

1 Comment

Raphvanns Over a year ago

Almost never a good idea to do 777 perms, especially on a jar.

Collectives™ on Stack Overflow

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hive/serde2/SerDe

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related