How to avoid Java OutofMemoryError Heap Space in loop with StringBuilder? [duplicate]

Question

Here is my code that throws java.lang.OutOfMemoryError: Java heap space:

StringBuilder sb = new StringBuilder();
    for (int i = 0; i< ranks.size(); i++) {
        Map<String, String> map = ranks.get(i);
        sb.append("SELECT ");
        int iter = 0;
        for (String key : map.keySet()) {
            if (key.equals("ware_md5")) {
                sb.append("'").append(map.get(key)).append(" ' AS ").append(key);
            }
            else {
                sb.append(map.get(key)).append(" AS ").append(key);
            }
            if (iter < map.keySet().size() - 1) {
                sb.append(",");
            }
            iter++;
        }
        if (i < ranks.size() - 1) {
            sb.append(" UNION ALL ");
        }

    }
    return sb.toString();

'ranks' ArrayList contains about 30 000 values

Is there any way to rewrite this peace of code to avoid this error ?

What are you going to do with a string this big? Maybe you should write directly to a file? What is the size of the map.keySet()? — 9000
– 9000, Commented Jun 6, 2017 at 20:46
Here I am trying ti generate some type of SQL request, size of map.keySet() is about 220 records — Bogdan Timofeev
– Bogdan Timofeev, Commented Jun 6, 2017 at 20:50
There are number of issues that you may run into even if you resolved memory issue. 1 big query that the db might not be able to handle. Large number of unions which performance will be a big hit on db server as well as the number of rows returned. I would suggest do-less-but-do-more approach — Minh Kieu
– Minh Kieu, Commented Jun 6, 2017 at 20:53
I'm afraid an SQL request size is limited in most databases by some reasonably small amount, like 1MB in MySQL (1GB in Postgres, though). Even if you can send a query this big, chances are executing it will face certain difficulties. Are you sure you want to query 30k tables in exactly one query? Chances are that running several dozens or hundreds of smaller queries in parallel, using a ThreadPoolExecutor, may result in better overall performance. You can easily implement the effect of union all on the receiving side. — 9000
– 9000, Commented Jun 6, 2017 at 20:59

user3745362 · Accepted Answer · 2017-06-06 20:46:00Z

4

Launch the JVM with a larger heap. java -Xmx256M as an example, for an example with a 256MB heap.

answered Jun 6, 2017 at 20:46

user3745362

2291 silver badge5 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Joop Eggen · Accepted Answer · 2017-06-06 20:51:05Z

1

More memory is the easiest solution, already given
new StringBuilder(100_000) - at least faster
AS key is not needed but for the first union.
Note that SortedMap is what you need: the keys always in the same order.
consume ranks.get(i)

answered Jun 6, 2017 at 20:51

Joop Eggen

110k8 gold badges89 silver badges142 bronze badges

Collectives™ on Stack Overflow

How to avoid Java OutofMemoryError Heap Space in loop with StringBuilder? [duplicate]

2 Answers 2

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Linked

Related