Spring Batch - Validate Header Lines in input csv file and skip the file if it invalidates

Question

I have a simple job as below:

<batch:step id="step">
 <batch:tasklet>
  <batch:chunk reader="itemReader" processor="itemProcessor" writer="itemWriter" commit-    interval="5000" />
 </batch:tasklet>
</batch:step>

itemReader is as below:

<bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
 <property name="linesToSkip" value="1"></property>
 <property name="skippedLinesCallback" ref="skippedLinesCallback" ></property>

 <property name="lineMapper">
    <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <property name="lineTokenizer" ref="lineTokenizer">                
                <property name="delimiter" value="," />                    
            </bean>
        </property>
        <property name="fieldSetMapper">
            <bean
                class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper" />
        </property>
    </bean>
</property>
<property name="resource" value="#{stepExecutionContext['inputKeyName']}" />
</bean>

<bean id"lineTokenizer"                        class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">

<bean id="skippedLinesCallback" class="com.test.IteMReaderHeader" >
<property name="lineTokenizer" ref="lineTokenizer">
</bean>

I am setting the "names" of the input fields in "com.test.IteMReaderHeader" class by injecting "lineTokenizer" in it.

I need to validate the header lines which is the 1st line in the input csv file with a fixed header value and if the header line invalidates then in that case I need to fail the step and skip the entire file so that the next file can be used for reading.

Please suggest a suitable way of achieving it. I would really appreciate your time and valuable inputs.

Thanks !!

Luca Basso Ricci · Accepted Answer · 2013-08-29 21:46:55Z

4

Looking code of FlatFileItemReader file stop condition is managed;

with private field boolean noInput
with private function readLine() used in protected doRead()

IMHO the best solution is to throw a runtime exception from your skippedLineCallback and manage error as reader exhaustion condition.

Foe example writing your delegate in this way

class SkippableItemReader<T> implements ItemStreamReader<T> {
  private ItemStreamReader<T> flatFileItemReader;
  private boolean headerError = false;

  void open(ExecutionContext executionContext) throws ItemStreamException {
    try {
      flatFileItemReader.open(executionContext);
    } catch(MyCustomExceptionHeaderErrorException e) {
      headerError = true;
    }
  }

  public T read() {
    if(headerError)
      return null;
    return flatFileItemReader.read();
  }

  // Other functions delegation
}

(you have to register delegate as stream manually,of course)
or extending FlatFileItemReader as

class SkippableItemReader<T> extends FlatFileItemReader<T> {
  private boolean headerError = false;

  protected void doOpen() throws Exception {
    try {
      super.doOpen();
    } catch(MyCustomExceptionHeaderErrorException e) {
      headerError = true;
    }
  }

  protected T doRead() throws Exception {
    if(headerError)
      return null;
    return super.doRead();
  }    
}

The code has been written directly without test so there can be errors, but I hope you can understand my point.
Hope can solve your problem

edited Aug 29, 2013 at 21:46

answered Aug 29, 2013 at 19:46

Luca Basso Ricci

18.5k2 gold badges51 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

grathi Over a year ago

Thanks for your inputs. The 2nd approach (extending FlatFileItemReader) is working fine. There were some issues while using the 1st approach (i.e implements ItemStreamReader<T>) Also, in the above scenario is it possible to fail the step but still skip the file ?

Luca Basso Ricci Over a year ago

Maybe mantaining headerError value in step (or job) execution context and manage it in doOpen()/doUpdate()/doClose() or with a listener. I have never tried that, sorry.

Collectives™ on Stack Overflow

Spring Batch - Validate Header Lines in input csv file and skip the file if it invalidates

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related