I want to make an array of size 10^9 elements where each elements can be an integer of the same size. I always get an OutOfMemoryException at the initialization line. How can I achieve this?
If this is not possible, please suggest alternative strategies?
Arrays are limited to 2GB in .net 4.0 or earlier, even in a 64 bit process. So with one billion elements, the maximum supported element size is two bytes, but an int is four bytes. So this will not work.
If you want to have a larger collection, you need to write it yourself, backed by multiple arrays.
In .net 4.5 it's possible to avoid this limitation, see Jon Skeet's answer for details.
<gcAllowVeryLargeObjects> element."Assuming you mean int as the element type, you can do this using .NET 4.5, if you're using a 64-bit CLR.
You need to use the <gcAllowVeryLargeObjects> configuration setting. This is not on by default.
If you're using an older CLR or you're on a 32-bit machine, you're out of luck. Of course, if you're on a 64-bit machine but just an old version of the CLR, you could encapsulate your "one large array" into a separate object which has a list of smaller arrays... you could even implement IList<int> that way so that most code wouldn't need to know that you weren't really using a single array.
(As noted in comments, you'll still only be able to create an array with 231 elements; but your requirement of 109 is well within this.)
I think you shouldn't load all this data into memory, store it somewhere in a file, and make a class that could work as an array but actually reads and writes data from and in the file
this is the general idea (this won't work as it is of course, plus you'll have to convert your int values into byte[] array before writing it and some other things)
public class FileArray
{
Stream s;
public this[int index]
{
get { s.Position = index * 4; return s.Read(); }
set { s.Position = index * 4; s.Write(value); }
}
}
that way you would have something that works just like an array, but the data would be stored on your hard drive
