2

I am a newer to C#. Recently when I update an application(vs2008), I met the following problem.

The application has c++ function helper as follows:

array<float>^ Variant::CopyToFloats()
{
   unsigned int n = this->data_uint8->Length;
   array<float>^ dst = gcnew array<float>(n); //<<OutOfMemoryException happened here
   for (unsigned int i = 0; i < n; i++)
    dst[i] = (float)this->data_uint8[i];
   return dst;
}

In c# file, 

    for(int i=0; i<m; i++)
    {  for(int j=0; j<n; j++)
       {
         float[] scan = data[i].CopyToFloats();
         for(int k=0; k<nn; k++)
           sample[k]=scan[function(i,j)];
       }
   }

When I run the application, OutOfMemoryException happens.

Then I added the following code 

   Process proc = Process.GetCurrentProcess();
   long memory = proc.PrivateMemorySize64;

before and after the outer loop, I found that the memory of scan was not released.

I tried the following ways:

1.Clear scan and set it to null, with/without using GC.Collect()

for(int i=0; i<m; i++)
{  for(int j=0; j<n; j++)
   {
     float[] scan = data[i].CopyToFloats();
     for(int k=0; k<nn; k++)
       sample[k]=scan[function(i,j)];
    }
    Array.Clear(scan, 0, scan.Length);
    scan = null;
    //GC.Collect();
}

With calling GC.Collect(), the program ran very slowly. Without calling, the program still crashed as OOME.

I was wondering which memory is not released? scan or array created by gcnew?

2.As the array size is big(>500000), I allocate big size array before entering the loop.

    float[] scan = new float[data[0].GetSize()];
    for(int i=0; i<m; i++)
    {  for(int j=0; j<n; j++)
       {
          scan = data[i].CopyToFloats();
          for(int k=0; k<nn; k++)
             sample[k]=scan[function(i,j)];
       }
   }

But OOME still happened. From here, I am kind of sure that the memory of array created by gcnew was not released. Am I right? If I am right, why it was not released? Is there any way to release this memory? If I am not right, please give me some advice, thanks!

Improve this question

2 Answers 2

Reset to default
1

Maybe it would be easier not to convert an entire array to float but just cast value you need

for(int i=0; i<m; i++)
{  for(int j=0; j<n; j++)
   {
     byte[] scan = data[i];
     for(int k=0; k<nn; k++)
       sample[k]=(float)scan[function(i,j)];
   }
}

It doesn't answers your question directly but can remove the need of using too much memory

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

It is possible that you are running into large object heap fragmentation. This would explain the out-of-memory errors you get, but I am not sure how fragmentation could happen if you only allocate arrays with identical sizes.

Pavel's suggestion to only cast each element as needed sounds like the best choice. If that is not possible or desired in your case, then preallocating the large array is a good idea, but there seems to be a bug in your posted example:

float[] scan = new float[data[0].GetSize()];
for(int i=0; i<m; i++)
{  for(int j=0; j<n; j++)
   {
      temp = data[i].CopyToFloats(); // you still allocate a new array here!
      for(int k=0; k<nn; k++)
         sample[k]=scan[function(i,j)]; // scan has not been updated!
   }
}

Instead of temp = data[i].CopyToFloats(); you need to do something like
data[i].CopyToFloats(scan). You need to change the signature of your c++ function to be able to use the preallocated array.

some additional ideas:

Just because the OutOfMemory Exception happens when you create a new array, this does not mean that the array is the leaking resource. It is quite possible that the array is succesfully cleaned up by the GC each time, but some other object is not.

Is data[] an Array of Variant, or is data actually a custom collection class with an indexer?
If it is, I would highly suspect the indexer to be the problem.

Does the program run without crash if you use GC.WaitForPendingFinalizers() instead of GC.Collect()?
If so, then your problem is some object with a finalizer clogging up the finalizer thread. This will happen if new objects are created quicker than they can be finalized. Every C++/cli class with a destructor is a candidate for that.

Improve this answer

5 Comments

The bug you mentioned is my typing error. I fixed it in my post. Thanks!!
The program still runs with crash with using GC.WaitForPendingFinalizers().
The way you changed the typing error still does not reuse the same array. CopyToFloats will create a new array each time. It does not matter if scan contained another array before that, it will not be reused or overwritten, you are only changing to which array the reference scan points. You need to pass the array you want to reuse as a parameter to CopyToFloats, and CopyToFloats has to use that array instead of creating another one with gcnew. Reusing the array is not possible without changing the c++ CopyToFloats function.
Really appreciate your reply. I modified the c++ function to void Variant::CopyToFloats(array<float>^ dst) {unsigned int n = this->data_uint8->Length; for (unsigned int i = 0; i < n; i++) dst[i] = (float)this->data_uint8[i]; } and modify C# code to: float[] scan = new float[data[0].GetSize()]; for(int i=0; i<m; i++) { for(int j=0; j<n; j++) { data[i].CopyToFloats(scan); for(int k=0; k<nn; k++) sample[k]=scan[function(i,j)];} } Then when i run the program, it runs very very slow. Is my way correct? I could not find useful tips with google.
that code looks correct to me. It should definitively not run slower than the original version; if it does something very weird is going on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.