0

i'm using weblogic and java ee. when i try to read a text file (about 100mb) and create an array out of it, java heap size exceeds and weblogic goes down. i use scanner to avoid reading file in memory and also try not to create any new variable inside the loop to prevent java heap error. but unfortunately i get that error and get weblogic exit on panic message in the console and it breaks down.

public static CensusData[] load(String path) throws Exception {
        List<CensusData> list = new ArrayList<>(43);
        String s;
        try (FileInputStream inputStream = new FileInputStream(path); Scanner sc = new Scanner(inputStream, "UTF-8")) {
            while (sc.hasNextLine()) {
                 s=sc.nextLine();
                list.add(new CensusData(Integer.parseInt(s.split(",")[0].trim()),
                        s.split(",")[1].trim(),
                        Integer.parseInt(s.split(",")[2].trim()),
                        Integer.parseInt(s.split(",")[3].trim()),
                        s.split(",")[4].trim(),
                        Integer.parseInt(s.split(",")[5].trim()),
                        s.split(",")[6].trim(),
                        s.split(",")[7].trim(),
                        s.split(",")[8].trim(),
                        s.split(",")[9].trim(),
                        s.split(",")[10].trim(),
                        s.split(",")[11].trim(),
                        s.split(",")[12].trim(),
                        s.split(",")[13].trim(),
                        s.split(",")[14].trim(),
                        s.split(",")[15].trim(),
                        Integer.parseInt(s.split(",")[16].trim()),
                        Integer.parseInt(s.split(",")[17].trim()),
                        Integer.parseInt(s.split(",")[18].trim()),
                        s.split(",")[19].trim(),
                        s.split(",")[20].trim(),
                        s.split(",")[21].trim(),
                        s.split(",")[22].trim(),
                        s.split(",")[24].trim(),
                        s.split(",")[25].trim(),
                        s.split(",")[26].trim(),
                        s.split(",")[27].trim(),
                        s.split(",")[28].trim(),
                        s.split(",")[29].trim(),
                        s.split(",")[30].trim(),
                        s.split(",")[31].trim(),
                        s.split(",")[32].trim(),
                        s.split(",")[33].trim(),
                        s.split(",")[34].trim(),
                        s.split(",")[35].trim(),
                        s.split(",")[36].trim(),
                        s.split(",")[37].trim(),
                        s.split(",")[38].trim(),
                        Integer.parseInt(s.split(",")[39].trim()),
                        s.split(",")[40].trim()));
            }
            // note that Scanner suppresses exceptions
            if (sc.ioException() != null) {
                throw sc.ioException();
            }
        } catch (Exception e) {
           e.printStackTrace();
        }

i know increasing Xmx size will help (which in my case will not), i'm looking for a real solution. why heap size is exceeded while i'm not storing any new instance inside the loop?

also consider the case. i need to get an array of CensusData out of the file. therefore i iterate each line of the text file and split it to get necessary data for creating that array.

4
  • You claim to not store any data in the loop. However, you are adding CensusData object to your list, thus storing data in memory. Commented Dec 7, 2021 at 7:06
  • i mean i'm not creating new instance of list inside the loop. just adding to the existed one. besides i have no choice. what can i do? Commented Dec 7, 2021 at 7:18
  • Store it in database? Commented Dec 7, 2021 at 13:53
  • no. i need to create an instance of CensusData out of each line of the text file. that's why i add to the list in each iteration Commented Dec 8, 2021 at 5:22

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.