How to sequentially write a stream in Node in a loop with writeable Streams?

Question

I want to append lines to a file, which is a writeable stream, in Node, in a specific order, through a loop.

Here is a simplified version of what I am talking about:

var fs = require('fs');
var outfile = fs.createWriteStream('outfile.txt');

for (var i = 0; i < 200; i++) {
    outfile.write('i', function (err, len) {
        console.log('written');
    });
}

I need the file to be written sequentially meaning that it would write, 12345...etc in order. My understanding is that outfile.write is async, meaning that it is possible for the file to be written out of order - is this correct?

I see from the node docs

Note that it is unsafe to use fs.write multiple times on the same file without waiting for the callback. For this scenario, fs.createWriteStream is strongly recommended.

So I believe that using a stream is best, however, I am not sure hot to guarantee that the data chunk was written succesfully before moving on to the next iteration in the loop. How should I go about doing that? Thanks.

inf3rno · Accepted Answer · 2021-02-04 16:31:55Z

0

I think with both the current streams and the streams you had in 2015 the write was never async parallel. Write streams have a buffer in memory and you write to that buffer not to the hard disk directly. The buffer will keep your sequence in the right order.

edited Feb 4, 2021 at 16:31

answered Feb 2, 2021 at 23:32

inf3rno

26.3k13 gold badges122 silver badges208 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Rūdolfs Vikmanis · Accepted Answer · 2015-11-18 04:26:56Z

-3

You can avoid using fs.createWriteStream by using fs.writeSync which doesn't have a callback problem like fs.write

answered Nov 18, 2015 at 4:26

Rūdolfs Vikmanis

7225 silver badges9 bronze badges

3 Comments

Startec Over a year ago

I am not a fan of this because of the synchronous and blocking nature of the code.

Rūdolfs Vikmanis Over a year ago

In your scenario, blocking is what is needed by definition of the problem.

Nulik Over a year ago

@RūdolfsVikmanis, the Writable object has been designed for writing sequential data. That's why it is called stream, so using writeSync() when you can take advantage of streaming functionality is foolish.

Collectives™ on Stack Overflow

How to sequentially write a stream in Node in a loop with writeable Streams?

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related