Reading Multiple Json Files And Converting To POJO Springboot batch

Question

I have a folder with thousands of text files with JSON content that I need to read, convert into a POJO and then save into a MySQL database. I intend to use a Spring Batch application.

Here is the issue so far, the research I have done only shows reading multiple CSV files or XML files and no JSON data. Specifically, I need to convert this method for parsing a CSV file into a JSON parser.

@Bean
public FlatFileItemReader<Person> reader() {
    FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
    reader.setLineMapper(new DefaultLineMapper<Person>() {{
        setLineTokenizer(new DelimitedLineTokenizer() {{
            setNames(new String[] {"firstname", "lastname", "email", "age"});
        }});
        setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
            setTargetType(Person.class);
        }});
    }});
    return reader;
}

This code parses a JSON file:

JSONParser parser = new JSONParser();
Object obj = parser.parse(new FileReader("C:\\path\\sample.json"));

The method might be something like this

@Bean
Public FileReader<Person> reader() {
    FileReader<Person> reader = new FileReader<Person>();

    /**** need help on what to do here ****/

    return reader;
}

Also seeing that I am reading all the files in a directory, I am passing the value of that directory in this format

@Value(value="C:\\path\\*.json")
private Resource[] resources;

So I need help on how to use this value (directory for all files) instead of what I showed earlier (single file location)

Object obj = parser.parse(new FileReader("C:\\path\\sample.json"));

what does private Resource[] resources; contains? does it contain array of each file path? — Ryuzaki L
– Ryuzaki L, Commented Jan 9, 2019 at 4:47

Mahmoud Ben Hassine · Accepted Answer · 2022-11-09 08:10:23Z

2

You can use the MultiResourceItemReader with a JsonItemReader as delegate. Here is a quick example:

@Bean
public JsonItemReader<Person> jsonItemReader(Resource[] resources) {
   JsonItemReader<Person> delegate = new JsonItemReaderBuilder<Person>()
      .jsonObjectReader(new JacksonJsonObjectReader<>(Person.class))
      .name("personItemReader")
      .build();

   MultiResourceItemReader<Person> reader = new MultiResourceItemReader<Person>();
   reader.setDelegate(delegate);
   reader.setResources(resources);
   return reader;
}

You can find more details about the JsonItemReader in the reference documentation.

Hope this helps.

edited Nov 9, 2022 at 8:10

answered Jan 9, 2019 at 21:57

Mahmoud Ben Hassine

32.1k5 gold badges38 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

jeanpic Over a year ago

The type should be Resource[]

Mahmoud Ben Hassine Over a year ago

@jeanpic Good catch! I updated the answer accordingly. Thank you.

Chris · Accepted Answer · 2019-10-14 06:05:46Z

0

https://docs.spring.io/spring-batch/4.0.x/reference/html/readersAndWriters.html#multiFileInput - I only found this after I found this answer

answered Oct 14, 2019 at 6:05

Chris

3991 silver badge14 bronze badges

Collectives™ on Stack Overflow

Reading Multiple Json Files And Converting To POJO Springboot batch

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related