Javascript - memory issues when dealing with large arrays

Question

I'm working on a Javascript app which does a lot of per-pixel operations on arrays which represent image data. When I'm setting up the arrays, I start with an empty '[]' and then fill it up with zeros, a bit like this:

<!DOCTYPE html>
<html>

  <head>

    <script>
      function makeBigArray()
      {
        // Create a data array for a 5000*5000 RGB image
        var imageData = [];
        for (var i = 0, lenI = 5000*5000*3; i < lenI; i++)
        {
            imageData.push(0);
        }
      }
    </script>

  </head>

  <body>

    <button onclick="makeBigArray()">Make Big Array</button>

  </body>

</html>

I noticed quite quickly that a loop like this causes what seems to be a massive memory hit. When I execute the above code, I get memory usages in the 2GB range.

Can someone explain why simply pushing elements into an array like that creates such high memory usage?

The array will scale dynamically as you put stuff in it. When it reaches a big enough size, it would need more memory allocated, so you'd get a whole new chunk of memory allocated, then the contents copied, then the old array would need to be garbage collected. — VLAZ
– VLAZ, Commented Oct 20, 2018 at 16:28
75 million numbers is about 600 megabytes, and system memory is allocated in chunks that get bigger as the desired (requested) storage space gets bigger. — Pointy
– Pointy, Commented Oct 20, 2018 at 16:33
It's easy to imagine what you are doing is similar to creating an RGB image, which would be about 24MB. That's what I thought when looked at this. What you are doing is different, however. Each pixel in an RGB image is 8-bits, but you are making an array of javascript numbers, which are significantly larger. — Mark
– Mark, Commented Oct 20, 2018 at 16:39
Just use typed arrays, you don't need that many bytes. developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays . Uint8 arrays are perfect for your scenario. — briosheje
– briosheje, Commented Oct 20, 2018 at 16:39

dthusian · Accepted Answer · 2018-10-22 20:20:43Z

1

In javascript, every number is a double, and on most systems is 8 bytes. That means your array is 600000000 bytes on most systems (600 MB). As well, most times when you push to your array, the browser allocates a new block of memory, and the garbage collector doesn't free the memory immediatly. If memory is a problem, use the Array constructor with one number, for the amount of elements, to perform less allocations. You could also use typed arrays, but they are CPU-intensive to modify.

edited Oct 22, 2018 at 20:20

answered Oct 20, 2018 at 16:37

dthusian

3784 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Bergi Over a year ago

"on most systems is 8 bytes" - JS numbers are always 64 bits (8 bytes)

Bergi Over a year ago

"every time you push to your array, the browser allocates a new block of memory" - no, that would be very inefficient. Resizable arrays are allocated using more clever strategies.

dthusian Over a year ago

@Bergi Fixed as I realized std::vector allocates more memory than it needs.

VLAZ Over a year ago

If memory is a problem, use the Array constructor with one number, for the amount of elements, to perform only one allocation

that would not allocate memory for all the members, though. Think about it - how would that even work? You can make an array for 100 items but they could be booleans or integers, or strings, or custom objects, or even mixed. You can also have sparse arrays. How much memory would you allocate for new Array(100)?

Pointy Over a year ago

5000 squared is 25,000,000 (25 million). Times 3 that's 75 million, and times 8 (size of a double) that's 600MB, not 60.

Collectives™ on Stack Overflow

Javascript - memory issues when dealing with large arrays

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related