How to record data that causes runtime exception in spring batch reader

Question

I need to capture the runtime exception in reader and skip them, so I use skipPolicy to do that as below. Also there is a requirement to record which data causes the runtime exception. I have some conversion logic in reader, but ItemReadListener does not have the access to item info. I am not sure where/how to access such data info.

Should I move the the conversion logic from reader to processor, then implement ItemProcessListener to record the error item? In this case, should the step1 method be udpated below? Thanks in advance!

@Bean
public Step step1() {
    return stepBuilderFactory.get("step1")
        .reader(reader1())
        .faultTolerant().skipPolicy(runtimeExceptionSkipper())
    .processor(processor1())
    .writer(writer1())
    .build();
}


@Bean(destroyMethod="")
public JdbcCursorItemReader<Job> reader1() {

    return new JdbcCursorItemReaderBuilder<Job>()
                    .dataSource(dataSourceConfig.dataSource())
                    .name("reader1")
                    .sql(Constants.QUERY_JOB_SQL)
                    .rowMapper(new JobRowMapper())
                    .build();

}

public class JobRowMapper implements RowMapper<Job> {


    public Job mapRow(ResultSet rs, int rowNum) throws SQLException {

        Job jobSchedule = new Job();
        String timeZone = rs.getString(Constants.COLUMN1);

        LocalDateTime localStart = 
        rs.getTimestamp(Constants.COLUMN2).toLocalDateTime();

        ZonedDateTime utcStart = ZonedDateTime.of(localStart, 
        ZoneId.of("GMT"));

        ZonedDateTime zonedStart = 
        utcStart.withZoneSameInstant(ZoneId.of(timeZone));

        job.setEffectDate(zonedStart.toLocalDate());

        return job;
    }
}

@Bean
public Step updatedStep1() {
    return stepBuilderFactory.get("updatedStep1")
        .reader(reader1())
    .processor(processor1())
        .faultTolerant().skipPolicy(runtimeExceptionSkipper())
        .listener(itemProcessListener())
    .writer(writer1())
    .build();
}

Michael Minella · Accepted Answer · 2019-04-12 17:41:04Z

1

The best you can do here is obtain the information from the Exception. The reason we don't provide it via a listener is that there is no "item" to pass on. Given that each source of the data is unique, we cannot provide a generic way to provide what that data was. However, in most cases, we do our best to provide as much information about what caused the error in the Exception. For example, the FlatFileParseException does include the String we were trying to parse as well as the line number where the error occurred. If you provide us with more information about the specific ItemReader you are using, we may be able to provide further guidance.

answered Apr 12, 2019 at 17:41

Michael Minella

21.6k4 gold badges61 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

kgb Over a year ago

thanks for the reply. I add the JdbcCursorItemReader in the code above. The runtime exception occurs in the rowmapper class where timezone is parsed due to the bad data. How can I record the bad data in such scenario? @Michael

Collectives™ on Stack Overflow

How to record data that causes runtime exception in spring batch reader

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related