Importing Multiple tables at same hdfs location using sqoop

Question

Is there any way to import multiple tables at a single HDFS location using sqoop.I Have 5 tables in db2 schema(tab1,tab2,tab3,tab4,tab5),and one hdfs location /user/hive/db2_tables. I want to store db2 data as below.

    /user/hive/db2_tables/tab1
    /user/hive/db2_tables/tab2
    /user/hive/db2_tables/tab3
    /user/hive/db2_tables/tab4
    /user/hive/db2_tables/tab5

but after importing first table when I'm giving same target directory in 2nd sqoop command its showing directory already exists. I'm using below sqoop command.

    sqoop import -Dmapreduce.output.basename="tab1" --connect jdbc:db2://xx.x.x.xxx:xxxxx/db2 --query 
    "select * from schemaname.tablename where  \$CONDITIONS " --username username -P --split-by id
     --target-dir /user/hive/db2_tables/ --fields-terminated-by '|'

Chema · Accepted Answer · 2020-09-25 05:53:29Z

1

Try to use warehouse-dir instead of target-dir argument.

With warehouse-dir you have to indicate a HDFS parent directory for table destination. For example: db2_tables, so first table will go to /user/hive/db2_tables/tab1, second table will go to /user/hive/db2_tables/tab2 and so on.

Your script would be as follow:

sqoop import -Dmapreduce.output.basename="tab1" \
--connect jdbc:db2://xx.x.x.xxx:xxxxx/db2 \
--query "select * from schemaname.tablename where  \$CONDITIONS " \
--username username -P \
--split-by id \
--warehouse-dir /user/hive/db2_tables/ \
--fields-terminated-by '|'

answered Sep 25, 2020 at 5:53

Chema

2,8282 gold badges17 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Importing Multiple tables at same hdfs location using sqoop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related