How to read CSV file with different number of columns with Spring Batch

Question

I have a CSV file that doesn't have a fixed number of columns, like this:

  col1,col2,col3,col4,col5    
  val1,val2,val3,val4,val5 
  column1,column2,column3
  value1,value2,value3

Is there any way to read this kind of CSV file with Spring Batch?

I tried to do this:

<bean id="ItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">

    <!-- Read a csv file -->
    <property name="resource" value="classpath:file.csv" />

    <property name="lineMapper">
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <!-- split it -->
            <property name="lineTokenizer">
                <bean
                    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="names"
                        value="col1,col2,col3,col4,col5,column1,column2,column3" />
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean
                    class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                    <property name="prototypeBeanName" value="myBean" />
                </bean>
            </property>

        </bean>
    </property>

</bean>

But the result was this error:

Take a look at the AbstractLineTokenizer#setStrict(boolean) (which DelimitedLineTokenizer inherits from) and set it to false. — fateddy
– fateddy, Commented Aug 4, 2017 at 9:27

Michael Minella · Accepted Answer · 2017-08-07 15:10:22Z

2

You can use the PatternMatchingCompositeLineMapper to delegate to the appropriate LineMapper implementation per line based on a pattern. From there, each of your delegates would use a DelimtedLineTokenizer and a FieldSetMapper to map the line accordingly.

You can read more about this in the documentation here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.html

answered Aug 7, 2017 at 15:10

Michael Minella

21.6k4 gold badges61 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

marie Over a year ago

Thank you very much @Michael

Gustavo Passini · Accepted Answer · 2018-09-01 02:59:52Z

2

AbstractLineTokenizer#setStrict(boolean) in your DelimitedLineTokenizer should do the job.

From the javadoc :

Public setter for the strict flag. If true (the default) then number of tokens in line must match the number of tokens defined (by Range, columns, etc.) in LineTokenizer. If false then lines with less tokens will be tolerated and padded with empty columns, and lines with more tokens will simply be truncated.

You should change this part of your configuration to:

<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
    <property name="names" value="col1,col2,col3,col4,col5,column1,column2,column3" />
    <property name="strict" value="false" />
</bean>

answered Sep 1, 2018 at 2:59

Gustavo Passini

2,70625 silver badges27 bronze badges

Collectives™ on Stack Overflow

How to read CSV file with different number of columns with Spring Batch

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related