2

Our application loads data from an avro file into in memory h2 database using java spring boot. I have a 145000000 rows to insert and I'm getting java heap error. it works fine with 2 million rows. How can I fix this issue?

Error pasted below Caused by: org.h2.jdbc.JdbcSQLNonTransientConnectionException: Out of memory.; SQL statement: at org.h2.message.DbException.getJdbcSQLException(DbException.java:617) ~[h2-1.4.199.jar:1.4.199] at org.h2.message.DbException.getJdbcSQLException(DbException.java:427) ~[h2-1.4.199.jar:1.4.199] at org.h2.message.DbException.get(DbException.java:194) ~[h2-1.4.199.jar:1.4.199] at org.h2.message.DbException.convert(DbException.java:341) ~[h2-1.4.199.jar:1.4.199] at org.h2.command.Command.executeUpdate(Command.java:278) ~[h2-1.4.199.jar:1.4.199] at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:233) ~[h2-1.4.199.jar:1.4.199] at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:205) ~[h2-1.4.199.jar:1.4.199] at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95) ~[HikariCP-3.2.0.jar:na] at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) ~[HikariCP-3.2.0.jar:na] at org.springframework.jdbc.datasource.init.ScriptUtils.executeSqlScript(ScriptUtils.java:488) ~[spring-jdbc-5.1.5.RELEASE.jar:5.1.5.RELEASE] ... 86 common frames omitted Caused by: java.lang.OutOfMemoryError: Java heap space

Improve this question
3
  • Can you insert rows in batch? 10000 by 10000 for example? Commented Mar 12, 2020 at 16:28
  • Which are your memory settings ? Increased them already ? Commented Mar 12, 2020 at 16:30
  • If it works with 2 million rows, that doesn't mean it will work with 145 million rows. Commented Mar 12, 2020 at 16:50

1 Answer 1

Reset to default
1

You're trying to load 145M rows into memory. So you obviously need to give the JVM more memory to handle that amount of storage.

Look into adding/changing your -Xmx JVM parameter to allocate more memory. See here for more info:

What are the -Xms and -Xmx parameters when starting JVM?

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I increased jvm memory and it worked
@Jewels: if possible, can you please post sample code? I am struggling with same issue with 30M records.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.