NiFi convert json to csv using ConvertRecord

Question

i have a stream of json in apache nifi that contain dynamic fields (maximum 11 fields) and i want to convert it to csv file.

sample json:

{  
   "field1":"some text",
   "field2":"some text",
   "field3":"some text",
   "field4":"some text",
   "field5":"some text",
   "field6":"some text",
   "field7":"some text"
}

i don't wanna using replace or json evaluate; how i do it with ConvertRecord? using this processor is so odd and hard to work...

Clear expression about dynamic fields: i have 11 fields at total. one record may have contain 7 fields, and next record may contain 11 fields and next 9 fields...

You are saying dynamic. Will the field names be changing then? — Sivaprasanna Sethuraman
– Sivaprasanna Sethuraman, Commented May 29, 2018 at 6:47
@sivaprasanna-sethuraman No, i have 11 fields. one record may have contain 7 fields, and next record may contain 11 fields and next 9 fields... — meh
– meh, Commented May 29, 2018 at 7:54

Sivaprasanna Sethuraman · Accepted Answer · 2018-05-29 08:59:03Z

The steps provided below will help you in getting this done.:

Connect your source processor which generates/outputs the JSON files to ConvertRecord.
Configure ConvertRecord and set 'Record Reader' to use JsonTreeReader controller service and 'Record Writer' to use CSVRecordSetWriter controller service
Configure both the controller services and set Schema Registry property to use AvroSchemaRegistry
Configure AvroSchemaRegistry. Go to 'Properties' tab and click the + button which lets you add a dynamic property.
Give some property name (ex: mySchema) and for the value, give the Avro schema expected for your input JSON. (You can use InferAvroSchema processor to generate Avro schema for your JSON)
Configure both JsonTreeReader and CsvRecordSetWriter and set the 'Schema Name' property to the name provided above, in this case, mySchema.
Connect the relationships of ConvertRecord to downstream processors according to your need.

Collectives™ on Stack Overflow

NiFi convert json to csv using ConvertRecord

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related