How to convert CSV to JSON using Apache Nifi?

Question

How to convert CSV to JSON using Apache Nifi?
I have a CSV file and I need to covert to json. How to draw a normal flow diagram. Please tell me the properties and figure details. My understanding is 4 block has to there. 1.GenerateFLow 2.ConverttoAvro 3.Converttojson 4.Savethefile. I am new to the Nifi and I have installed nifi, struggling to configure

"model","speed","mileage"
"audi",4,2
"benz",4,10
"bmw",7,4
"jaguar",7,22

My Json

[
  {
    "model": "audi",
    "speed": 4,
    "mileage": 2
  },
  {
    "model": "benz",
    "speed": 4,
    "mileage": 10
  },
  {
    "model": "bmw",
    "speed": 7,
    "mileage": 4
  },
  {
    "model": "jaguar",
    "speed": 7,
    "mileage": 22
  }
]

I went through Convert a CSV file to JSON using Apache NiFi

notNull · Accepted Answer · 2018-11-21 17:55:19Z

7

The approach that you have described in question is old (needs to follow if you are using prior to NiFi-1.2 version).

Starting from NiFi-1.2 introduced record oriented processors.

For your case Use ConvertRecord processor and Configure/enable Record Reader(CSVReader)/Writer(JsonSetWriter) controller services.
Then NiFi ConvertRecord processor reads the incoming CSV data and writes the output flowfile in JSON format.

Refer to this link describes step-by-step procedure how to convertCsvtoJson using ConvertRecord processor.

Starting from NiFi-1.2 Version Flow:

1.GenerateFlowFile 
2.ConvertRecord
3.SaveFile (Using PutFile/PutHDFS)

Prior to NiFi-1.2 Version Flow:

1.GenerateFlowFile
2.InferAvroSchema
3.ConvertCSVToAvro
4.ConvertAvroToJSON

edited Nov 21, 2018 at 17:55

answered Nov 21, 2018 at 17:20

notNull

31.8k4 gold badges41 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user1464878 Over a year ago

Just one query how upload the input file. Do we need to do Get FIle before the convert(step2)

notNull Over a year ago

@user1464878, Yes you can use either GetFile (or) ListFile + FetchFile processors to fetch input file from Local. pierrevillard.com/2017/02/23/…

Ak777 Over a year ago

Perfectly worked. The link provided solved my need. Thanks

CMPE · Accepted Answer · 2018-12-03 17:51:03Z

-3

You can create a python script that convert a CSV to JSON. Then add an 'ExecuteScript' processor to run this script with your flow. The Python script should be able to read the flow, convert CVS to JSON and then return the JSON into the flow. You can also use different programming language if you don't like Python.

answered Dec 3, 2018 at 17:51

CMPE

1,9735 gold badges27 silver badges41 bronze badges

Collectives™ on Stack Overflow

How to convert CSV to JSON using Apache Nifi?

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related