Spring batch: processing multiple record at once

Question

I am using spring batch and as normally used I have reader , processor and writer .

I have 2 questions

1> Reader queries all 200 records (total record size in table is 200 and I have given pagesize=200 )and thus it gets me all 200 records, and in processor we want list of all these record because we have to compare each record with other 199 records to group them in different tiers . Thus I am thinking if we can get that list in processing step , I can manipulate them .how should I approach .

2> In processing stage I need some master data from database depending on which all input records will be processed .i m thinking of injection of data source in processing bean and fetch all master table data and process all records. Is it good approach or please suggest otherwise .

<job id="sampleJob">
    <step id="step1">
        <tasklet>
            <chunk reader="itemReader" processor="processor" writer="itemWriter" commit-interval="20"/>
        </tasklet>
    </step>
</job>

And the processor is

@Override
public User process(Object item) throws Exception {
    // transform item to user
    return user;
}

And I want something like

public List<User> process(List<Object> item) throws Exception {
    // transform item to user
    return user;
}

I found some post here but they say to get the list in writer .But i dont like to process anything in writer, because that kills the defination of writer and processor. Is there any configuration to get the list inside this process method.

Thank you

Michael Minella · Accepted Answer · 2016-01-28 16:48:46Z

8

Since the ItemProcessor receives whatever you return from the ItemReader, you need your ItemReader to return the List. That List is really the "item" you're processing. There is an example of this in the Spring Batch Samples. The AggregateItemReader reads all the items from a delegate ItemReader and returns them as a single list. You can take a look at it on Github here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/java/org/springframework/batch/sample/domain/multiline/AggregateItemReader.java

answered Jan 28, 2016 at 16:48

Michael Minella

21.6k4 gold badges61 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

surya Over a year ago

Thanks . But this class is more of fetching records from file . Do we have similar class to fetch from database . As we can't override read method it's not possible to extend job paging item reader .

Michael Minella Over a year ago

The same idea applies and you don't override the read method on the pagingitemreader...you wrap that reader with one that reads from the delegate until the input is exhausted and then return that list as a single item.

surya Over a year ago

I created a delegate aggregate item reader , which seems working , I get a list of records in processing stage . But then it loops sith null values . I don't need to write any mapper and reader class , can u please confirm.

Chandresh Mishra Over a year ago

is AggregateItemReader part of spring batch artifact?

Sean Hwang Over a year ago

How do I know whether specific record is header/footer in a custom ItemReader when reading records from DB by pageSize each?

|

Collectives™ on Stack Overflow

Spring batch: processing multiple record at once

1 Answer 1

13 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

13 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related