class with many string variables causing memory problems

Question

Background: I have a 2 class A & B. Class A contains 1 int64, 1 int, 1 datetime, 6 strings, 1 byte[] and a list of Class B objects.

Class B contains 2 Int64, 1 enum (4values), 5 strings.

Class A and B have a master detail relationship between them. Class A can have 0 or more objects of Class B in its list.

These objects are used to store search results from our db.

We have a SOA architecture. The client can search withing a particular date range and with a fixed set of criteria. We use these on the service to gather the search results and push them to the client in batches. After sending each batch, we delete those objects from the service.

Problem: If a user does a search that returns 200k results (Class A + Class B counts combined), the service memory consumption jumps 200mb. But if subsequent searches return only few results, the memory doesn't come back to its previous state. When I look at the performance counters for the service, I see that lot of memory is taken up by gen2 objects.

Without trying to do a GC.Collect(), how can I make sure that that memory is returned more quickly? I though of using the IDisposable interface on Class A and Class B, but since most of the fields in both of them are strings, I don't know how I can forcefully dispose them.

Dylan Smith · Accepted Answer · 2011-10-04 16:11:37Z

7

You might want to consider attacking the problem from a different angle. Rather than trying to optimize low-level things like Garbage Collection details, why not take a step back and ask do you really need to return 200k results at once?

If this is being displayed in a UI then showing 200k results is completely useless to a human being as that's just way too much data. I'd suggest doing some kind of paging and only showing 100 at a time (and only retrieving 100 at a time from the service). Or maybe showing the user the first 100, then have them click a button if they really want to retrieve all 200k.

Improvements like that are going to have a much bigger impact than micro-optimizing memory management.

answered Oct 4, 2011 at 16:11

Dylan Smith

22.3k2 gold badges51 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Matt Over a year ago

completely agree - users don't want 200k rows of data

Jon Skeet · Accepted Answer · 2011-10-04 16:12:08Z

2

Even if you do collect the garbage, I don't think the process will return the memory to the operating system. Last time I looked, the CLR took the approach that if you'd used that much memory once, you're likely to need it again later.

On most modern machines, 200MB isn't really that much - is this definitely a real problem, or more of a concern?

answered Oct 4, 2011 at 16:12

Jon Skeet

1.5m893 gold badges9.3k silver badges9.3k bronze badges

Collectives™ on Stack Overflow

class with many string variables causing memory problems

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related