PutHBaseJSon Processor in Apache-Nifi

Question

I am using PutHBaseJSon processor that will fetch data from hdfs location and to put it into hbase.The data present in hdfs location is like below format and this is in a single file.

{"EMPID": "17", "EMPNAME": "b17", "DEPTID": "DNA"}            
{"EMPID": "18", "EMPNAME": "b18", "DEPTID": "DNA"}
{"EMPID": "19", "EMPNAME": "b19", "DEPTID": "DNA"}

when I execute the PutHBaseJSon processor, it's only fetching the first row and put it into the hbase table which I have created. Can't we able to fetch all the rows present in that file using this processor? or How to fetch all the records from the single file to hbase?

Bryan Bende · Accepted Answer · 2016-06-14 11:40:05Z

1

PutHBaseJSON takes a single JSON document as input. After fetching from HDFS, you should be able to use the SplitText processor with a line count of 1 to get each of your JSON documents into a single flow file.

If you have millions of JSON records in a single HDFS file, then you should perform a two phase split, the first SplitText should split with a line count of say 10,000 then then a second SplitText should split those down to 1 line each.

answered Jun 14, 2016 at 11:40

Bryan Bende

18.7k1 gold badge33 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Praneeth Gubba · Accepted Answer · 2021-06-16 07:47:54Z

-1

You can make use of SplitJson processor to split them as individual records serially they will be sent to puthbasejson

answered Jun 16, 2021 at 7:47

Praneeth Gubba

1

Collectives™ on Stack Overflow

PutHBaseJSon Processor in Apache-Nifi

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related