Convert JSON Lines to JSON array using Apache Nifi

Question

I have a file that includes (schemaless) JSON Lines encoded data.

For example:

{"foo" : "abc", "bar" : "def" }
{"foo" : "xyz" }
{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}

I would like to use NIFI to convert this into a JSON array:

[{"foo" : "abc", "bar" : "def" },{"foo" : "xyz" },{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}]

Andy · Accepted Answer · 2018-02-14 23:48:25Z

2

The easiest way to accomplish this in Apache NiFi is to use two ReplaceText processors. In the first, configure as:

Search Value: \}\s*\{
Replacement Value: \},\{
Replacement Strategy: Regex Replace
Evaluation Mode: Entire Text

This will remove the line breaks between the tuples and insert commas between them. In the second:

Search Value: (^.*$)
Replacement Value: [$1]
Replacement Strategy: Regex Replace
Evaluation Mode: Entire Text

This will add the enclosing brackets around the JSON array. There are other ways to accomplish this with ExecuteScript or JoltTransformJSON processors, but they are more complicated and brittle.

answered Feb 14, 2018 at 23:48

Andy

14.2k2 gold badges38 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Convert JSON Lines to JSON array using Apache Nifi

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related