I am iterating the Mongodb Collection with Java for loop which is creating heap Memory issue

Question

@RequestMapping(value = "/id/", method = RequestMethod.GET)
public String getClaimsDetailById(@RequestParam(value = "userId") String userId,
                                  @RequestParam(value = "id") String id) throws JsonProcessingException {
    MongoDatabase database = this.mongoClient.getDatabase(this.database);
    MongoCollection<Document> collection = database.getCollection(this.collection);
    List claim = new ArrayList();
    Document document = new Document("_id", new ObjectId(id));
    List<Document> claims = collection.find(document).into(claim);
    List<Document> claimsUpdatedList = new ArrayList<>();
    for (Document doc : claims) {
        if (null != doc.get("Common")) {
            Document common = (Document) doc.get("Common");
            if (null != common.get("EffectiveDate")) {
                Date date = (Date) common.get("EffectiveDate");
                common.put("EffectiveDate",convertDate(date));
            }
            if (null != common.get("ExpirationDate")) {
                Date date = (Date) common.get("ExpirationDate");
                common.put("ExpirationDate",convertDate(date));
            }
            doc.put("Common",common);
            claimsUpdatedList.add(doc);
        }
    }
    JsonWriterSettings writerSettings = JsonWriterSettings.builder().outputMode(JsonMode.SHELL).indent(true).build();
    return claimsUpdatedList.get(0).toJson(writerSettings);
}

Hi, I've formatted your code. However, a code dump is not a good question. Please edit your question and describe in plain English what problem you have. What is the memory issue? Do you have a stack trace to include? See also How to Ask. — Robert
– Robert, Commented May 26, 2020 at 15:01

Piotr Wilkin · Accepted Answer · 2020-05-26 16:20:39Z

2

The problem is that you're using into(), which loads all the data at once into the List, therefore possibly overloading the heap if the data is very large.

Instead, you should be using iterator(), which uses the database cursor to process the data one-by-one, without loading them into application memory. This concept is not restricted to Mongo, it is a common mechanism with all databases (which is why the very concept of cursors exists).

For example:

...
FindIterable<Document> documentCursor = collection.find(document);
for (Document doc : documentCursor) {
    ...
}

edited May 26, 2020 at 16:20

answered May 26, 2020 at 13:22

Piotr Wilkin

3,52112 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rose Over a year ago

Thank you valuable suggestion .Could you show one scenario as I am new to Mongodb

Piotr Wilkin Over a year ago

Added a sample modification to your code in the answer

Rose Over a year ago

Thank you for your response .It helped me .

Collectives™ on Stack Overflow

I am iterating the Mongodb Collection with Java for loop which is creating heap Memory issue

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related