How to convert a string to a JSON array using NiFi

Question

In NiFi I'm processing a flowfile containing the following attribute:

Key: 'my_array'
    Value: '[u'firstElement', u'secondElement']'

I'd like to split flowFile on this array to process each element separately (and then merge). I tried to use SplitJson processor, but it requires JSON content to operate on, so I used AttributesToJSON before it. Unfortunately the produced flowFile's content is:

{"my_array": "[u'firstElement', u'secondElement'"}

And I receive the error

The evaluated value [u'firstElement', u'secondElement'] of $['my_array'] was not a JSON Array compatible type and cannot be split.

Is it possible to convert my_array string to the correct JSON array? Do I need to use ExecuteScript or is there some simpler way?

the [u'firstElement', u'secondElement'] has not a valid json format. i think it's a kind of python formatting. — daggett
– daggett, Commented Jun 26, 2019 at 8:51
Yes, these values come from Python, but I think that the main problem is having "[u'firstElement', u'secondElement']" instead of [u'firstElement', u'secondElement'] — trivelt
– trivelt, Commented Jun 26, 2019 at 8:53
with doublequotes - it's just a string. without - not a valid json. could you produce json from python? — daggett
– daggett, Commented Jun 26, 2019 at 9:21
I tried to convert it to the list in Python, but it causes error org.python.core.PyList cannot be cast to java.lang.String , so it seems that Python script has to return string. — trivelt
– trivelt, Commented Jun 26, 2019 at 9:51
use json.dumps(...) to convert python data to json-formatted string — daggett
– daggett, Commented Jun 26, 2019 at 10:06

Ben Yaakobi · Accepted Answer · 2019-06-26 08:50:47Z

3

How about ReplaceText with Replacement Strategy of Always Replace and Replacement Value of ${my_array} and then SplitJSON? This will replace your FlowFile's content with this attribute's value and then you could SplitJSON on it.

answered Jun 26, 2019 at 8:50

Ben Yaakobi

1,67811 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

trivelt Over a year ago

Thank you, I used your solution along with the one proposed by @dagett. I changed Python script to set json.dumps(...) as attribute's value and than I put ReplaceText withReplacement Value = {"my_array": ${my_array}}

smbanaei · Accepted Answer · 2020-10-17 16:45:34Z

0

Suppose I want to string : "Hashtags": "['tag1','tag2']" (as part of my resultant json in Nifi,) to be changed into : "Hashtags": ['tag1','tag2'].

what I do is :

I use ReplaceText with Replacement Strategy : Regex Replace and Replacement Value : a regex Expression. This will replace FlowFile's matched content with this attribute's value and then you could continue your process.

answered Oct 17, 2020 at 16:45

smbanaei

1,1878 silver badges16 bronze badges

1 Comment

Wyllys Over a year ago

That does indeed work, thanks for the tip. It sucks that NiFi doesn't properly format JSON lists in AttributesToJson processor, instead it turns them into strings.

Collectives™ on Stack Overflow

How to convert a string to a JSON array using NiFi

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related