I have a partitioned Spring Batch job that reads several split up CSV files and processes each in their own thread, then writes the results to a corresponding output file.
If an item fails to process though (an exception is thrown), I want to write that result to an error file. Is there a way to add a writer or listener that can handle this?
Taking this one step further, is there a way to split this up by exception type and write the different exceptions to different files?
You can achieve this by specifying SkipPolicy. Implement this interface and add your own logic.
public class MySkipper implements SkipPolicy {
@Override
public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {
if (exception instanceof XYZException) {
//doSomething
}
......
}
You can specify this skip policy in your batch.
this.stepBuilders.get("importStep").<X, Y>chunk(10)
.reader(this.getItemReader()).faultTolerant().skipPolicy(....)
.processor(this.getItemProcessor())
.writer(this.getItemWriter())
.build();
One way that I have seen this done is through a combination of a SkipPolicy and a SkipListener.
The policy would allow you to skip over items that threw an exception, such as a FlatFileParseException (skippable exceptions can be configured).
The listener gives you access to the Throwable and the item that caused it (or just Throwable in the case of reads). The skip listener also lets you differentiate between skips in the read/processor/writer if you wanted to handle those separately.
public class ErrorWritingSkipListener<T, S> implements SkipListener<T, S> {
@Override
public void onSkipInRead(final Throwable t) {
// custom logic
}
@Override
public void onSkipInProcess(final T itemThatFailed, final Throwable t) {
// custom logic
}
@Override
public void onSkipInWrite(final S itemThatFailed, final Throwable t) {
// custom logic
}
}
I would recommend using the SkipPolicy only to identify the exceptions you want to write out to your various files, and leveraging the SkipListener to perform the actual file writing logic. That would match up nicely with their intended use as defined by their interfaces.
