Node.js encrypts large file using AES

Question

I try to use following code to encrypt a file of 1 GB. But Node.js abort with "FATAL ERROR: JS Allocation failed - process out of memory". How can I deal with it?

var fs = require('fs');
var crypto = require('crypto');
var key = "14189dc35ae35e75ff31d7502e245cd9bc7803838fbfd5c773cdcd79b8a28bbd";
var cipher = crypto.createCipher('aes-256-cbc', key);
var file_cipher = "";
var f = fs.ReadStream("test.txt");
f.on('data', function(d) {
    file_cipher = file_cipher + cipher.update(d, 'utf8', 'hex');
});
f.on('end', function() {  
    file_cipher = file_cipher + cipher.final('hex');
});

mscdex · Accepted Answer · 2016-11-26 20:45:12Z

50

You could write the encrypted file back to disk instead of buffering the entire thing in memory:

var fs = require('fs');
var crypto = require('crypto');

var key = '14189dc35ae35e75ff31d7502e245cd9bc7803838fbfd5c773cdcd79b8a28bbd';
var cipher = crypto.createCipher('aes-256-cbc', key);
var input = fs.createReadStream('test.txt');
var output = fs.createWriteStream('test.txt.enc');

input.pipe(cipher).pipe(output);

output.on('finish', function() {
  console.log('Encrypted file written to disk!');
});

edited Nov 26, 2016 at 20:45

answered Dec 7, 2014 at 18:04

mscdex

107k15 gold badges201 silver badges159 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Jan Leo Over a year ago

Thanks! BTW, how can I know when the encryption is done?

mscdex Over a year ago

You can listen for the finish event on output.

sarath Over a year ago

How to decrypt the same file?

Ryan Plant Over a year ago

You can decrypt in the exact same way by using crypto.createDecipher instead of crypto.createCipher.

SamB Over a year ago

createCipher is deprecated

|

Ihor Sakailiuk · Accepted Answer · 2022-01-05 16:05:31Z

14

crypto.createCipher() without initialization vector is deprecated since NodeJS v10.0.0 use crypto.createCipheriv() instead.

You can also pipe streams using stream.pipeline() instead of pipe method and then promisify it (so the code will easily fit into promise-like and async/await flow).

const {createReadStream, createWriteStream} = require('fs');
const {pipeline} = require('stream');
const {randomBytes, createCipheriv} = require('crypto');
const {promisify} = require('util');

const key = randomBytes(32); // ... replace with your key
const iv = randomBytes(16); // ... replace with your initialization vector

promisify(pipeline)(
        createReadStream('./text.txt'),
        createCipheriv('aes-256-cbc', key, iv),
        createWriteStream('./text.txt.enc')
)
.then(() => {/* ... */})
.catch(err => {/* ... */});

With NodeJS 15+ you could simplify it (skip promisify part)

const {createReadStream, createWriteStream} = require('fs');
const {pipeline} = require('stream/promises');
const {randomBytes, createCipheriv} = require('crypto');

const key = randomBytes(32); // ... replace with your key
const iv = randomBytes(16); // ... replace with your initialization vector

pipeline(
  createReadStream('./text.txt'),
  createCipheriv('aes-256-cbc', key, iv),
  createWriteStream('./text.txt.enc')
)
.then(() => {/* ... */})
.catch(err => {/* ... */});

edited Jan 5, 2022 at 16:05

answered Jun 17, 2020 at 17:06

Ihor Sakailiuk

6,1483 gold badges28 silver badges39 bronze badges

3 Comments

Prathamesh More Over a year ago

Me getting this error while decrypting the file Error: Unsupported state or unable to authenticate data

Ihor Sakailiuk Over a year ago

@PrathameshMore It's difficult to say where the issue is without seeing the code for file decryption. Please note, you should store your key and initialization vector somewhere (e.g. file), randomBytes was used only for demonstration purpose.

Felix Lechenbauer Over a year ago

Stream Promises API is available since Node Version 15 (docu)

Lee Goddard · Accepted Answer · 2023-03-28 09:07:02Z

3

I would simply use the Fileger to do it. It's a promise-based package and a clean alternative to the NodeJS filesystem.

const fileger = require("fileger")
const file = new fileger.File("./your-file-path.txt");

file.encrypt("your-password") // this will encrypt the file
   .then(() => {
      file.decrypt("your-password") // this will decrypt the file
   })

edited Mar 28, 2023 at 9:07

Lee Goddard

11.3k5 gold badges53 silver badges71 bronze badges

answered Apr 18, 2020 at 11:23

Paul Hanneforth

732 silver badges7 bronze badges

Collectives™ on Stack Overflow

Node.js encrypts large file using AES

3 Answers 3

8 Comments

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related