Spring Data JPA not saving to database when using Spring Batch

Question

I have a Spring Boot application using JPA that has 1 PostgreSQL database. And I am using Spring Batch. The scenario is I am reading a file and writing data to the PostgreSQL database. The program works with PostgreSQL when it is creating meta-data tables used by Spring Batch in the database. But what I need is Spring Boot not to create meta-data tables and use in-memory Map-based job repository by Spring Batch. I don't want to create meta-data tables in the database at all. Need to just execute in-memory Map-based. I tried many answers but none of them worked. And I see,

MapJobRepositoryFactoryBean is deprecated

Here is my BatchConfiguration.java class,

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {

    private @Autowired JobBuilderFactory jobBuilderFactory;
    private @Autowired StepBuilderFactory stepBuilderFactory;
    private @Autowired UserItemReader userItemReader;
    private @Autowired UserItemProcessor userItemProcessor;
    private @Autowired UserItemWriter userItemWriter;

    @Bean
    public Step importUsersStep() {
        return stepBuilderFactory.get("STEP-01")
                .<User, User>chunk(10)
                .reader(userItemReader)
                .processor(userItemProcessor)
                .writer(userItemWriter)
                .build();
    }

    @Bean
    public Job importUsersJob() {
        return jobBuilderFactory.get("JOB-IMPORT")
                .flow(importUsersStep())
                .end()
                .build();
    }
}

Repository.java class,

public interface UserRepository extends JpaRepository<User, Long> {

}

And in UserItemWriter.java in write method I am calling userRepository.saveAll();

So how can I make this to work with in-memory Map-based and also since I am using Spring Data JPA to save the user to the database so the save should also work without any issue because when I am trying some approaches I saw nothing committed to PostgreSQL and I think even the user data also committed to in-memory Map. So anybody can help me? Thanks in advance.

Which transaction manager are you using? By default, Spring Batch uses a DataSourceTransactionManager if you provide a datasource in your context. But if you use JPA, you need to configure Spring Batch to use a JpaTransactionManager instead. — Mahmoud Ben Hassine
– Mahmoud Ben Hassine, Commented Nov 23, 2020 at 8:51

Shawrup · Accepted Answer · 2020-11-22 16:43:23Z

2

We solve this problem by defining multiple datasources. One in memory datasource for Spring Batch Metadatas and another or more datasources for jobs depending on configuration.

# for spring batch metadata
spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password

# for jobs
job.datasource.url=jdbc:postgresql://localhost:5432/postgres
job.datasource.username=user
job.datasource.password=password

We create datasource manually like

    @Bean
    @Primary
    @ConfigurationProperties("spring.datasource")
    public DataSourceProperties mysqlDataSourceProperties() {
        return new DataSourceProperties();
    }

and another one.

@Configuration
@EnableJpaRepositories(basePackages = "your.package.name",
        entityManagerFactoryRef = "postgresEntityManagerFactory",
        transactionManagerRef= "postgresTransactionManager")
public class PostgresDataConfig {

    @Bean
    @ConfigurationProperties("postgres.datasource")
    public DataSourceProperties postgresDataSourceProperties() {
        return new DataSourceProperties();
    }

    @Bean(name = "postgresEntityManagerFactory")
    public LocalContainerEntityManagerFactoryBean postgresEntityManagerFactory(
            EntityManagerFactoryBuilder builder) {
        return builder
                .dataSource(postgresDataSource())
                .packages("com.bimurto.multipledatasource.postgres")
                .build();
    }

    @Bean(name = "postgresTransactionManager")
    public PlatformTransactionManager postgresTransactionManager(
            final @Qualifier("postgresEntityManagerFactory") LocalContainerEntityManagerFactoryBean entityManagerFactory) {
        return new JpaTransactionManager(Objects.requireNonNull(entityManagerFactory.getObject()));
    }
}

This allows to repositories use another datasource than the default one.

answered Nov 22, 2020 at 16:43

Shawrup

2,8142 gold badges15 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Abraham Arnold Over a year ago

Thank you very much for the answer and your time.

fuat Over a year ago

Did you add @Transactional("postgresTransactionManager") to your methods? Because I have the same case, even if I add H2 DB the transaction is managed by spring-batch, but the job data source was primary.

Sergi Almar · Accepted Answer · 2020-11-22 11:32:43Z

1

MapJobRepositoryFactoryBean was only recommended for testing and development purposes, not for production use, it has a number of issues, that's why it was deprecated.

Spring Batch will try to use the DataSource from your ApplicationContext if you have one. If you don't want Spring Batch to use it to store its state, you could use a JobRepositoryFactoryBean with an embedded database. The easiest way to do so, is to define a BatchConfigurer bean:

@Bean
public BatchConfigurer configurer() {
    return new DefaultBatchConfigurer(dataSource());
}

public DataSource dataSource(){
    return new EmbeddedDatabaseBuilder()
            .setType(EmbeddedDatabaseType.HSQL)
            .addScript("classpath:org/springframework/batch/core/schema-drop-hsqldb.sql")
            .addScript("classpath:org/springframework/batch/core/schema-hsqldb.sql")
            .build();
}

answered Nov 22, 2020 at 11:32

Sergi Almar

8,4443 gold badges36 silver badges32 bronze badges

3 Comments

Abraham Arnold Over a year ago

I tried this code. But it's not working it's not saving data to database may be it's saving to in-memory database. Do I need to configure anything with JobRepositoryFactoryBean . I am sorry for my lack of knowledge in this as I am new to this. So kindly can you help me?

Sergi Almar Over a year ago

The DefaultBatchConfigurer will configue a JobRepositoryFactoryBean with the passed dataSource. You should have your own PostgreSQL datasource defined as a bean (which will be used by your JPA repositories)

Abraham Arnold Over a year ago

I am currently defining all things in the properties file. Including datasource url for PostgreSQL. Ho can I configure that to work on this?

Collectives™ on Stack Overflow

Spring Data JPA not saving to database when using Spring Batch

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related