Problem with 'Sort exceeded memory limit' MongoDb in Spring

Question

I'm writing a method to implement a query using MongoRepository, I'm using aggregation in a Java/Spring project, but it is giving exceeded memory limit when I'm testing.

I've tried use

newAggregationOptions().cursorBatchSize(pageable.getPageSize()).allowDiskUse(true).build()

but didn't work

my method:

...
        var matchCriteria = match(criteria);

        var unwindVariations = unwind("variations", false);


        var sort = sort(Sort.Direction.ASC, "variations.plataformStatus.status");

        var countResult = count().as("totalElements");
        var aggregationOptions = Aggregation.newAggregationOptions().cursorBatchSize(pageable.getPageSize()).allowDiskUse(true).build();

        var countAggregation = newAggregation(
                matchCriteria,
                unwindVariations,
                sort,
                countResult)
                .withOptions(aggregationOptions);

        var totalElements = mongoTemplate.aggregate(countAggregation, "PRODUCT", Map.class).getMappedResults();

error message:

com.mongodb.MongoCommandException: Command failed with error 16819 (Location16819): 'Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.' on server cluster0-shard-00-01-du380.mongodb.net:27017.

Could you identify if the Exception is thrown in the first or second aggregation operation? — Gonzalo Matheu
– Gonzalo Matheu, Commented Jun 6, 2019 at 12:56

Gonzalo Matheu · Accepted Answer · 2019-06-06 12:27:27Z

2

You should enable allowDiskUse option.

By default, sort operation has a 100MB limit (in memory), enabling that option in your operation will allow to use bigger size.

From MongoDB's sort documentation:

The $sort stage has a limit of 100 megabytes of RAM. By default, if the stage exceeds this limit, $sort will produce an error. To allow for the handling of large datasets, set the allowDiskUse option to true to enable $sort operations to write to temporary files.

answered Jun 6, 2019 at 12:27

Gonzalo Matheu

10.2k6 gold badges42 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Pedro Henrique Over a year ago

I've tried use newAggregationOptions().cursorBatchSize(pageable.getPageSize()).allowDiskUse(true).build() What's wrong? is already enable

Collectives™ on Stack Overflow

Problem with 'Sort exceeded memory limit' MongoDb in Spring

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related