I have to be able to read a large input from stdin (10^5 space delimited numbers). Any input above 10^3 and readline takes almost 200 seconds just to read it, I need to be able to do so in less than five seconds. Should I be using something other than readline or is there a way to increase readline's buffer or something?
2 Answers
readline, like it says on the tin, is designed to read lines. Since your space-separated numbers are (presumably) all on one very long line, it's not surprising readline's having trouble.
This would be pretty easy to solve with a Transform stream, but you don't have to because someone's already done the work for you. Take a look at the split2 module, which takes a stream of data and splits it on a given delimiter. You would use it like this:
const split2 = require('split2');
process.stdin
.pipe(split2(' '))
.pipe(process.stdout);
On my machine (MacBook Pro, Intel Core i7), the above takes 0.41s for 100,000 space-separated numbers and 2.06s for 1,000,000 numbers.
Of course, that's terribly unuseful. Suppose we wanted to sum the numbers instead:
const split2 = require('split2');
let sum = 0;
process.stdin.pipe(split2(' '))
.on('data', data => sum += parseInt(data, 10))
.on('end', () => console.log('Sum: %d', sum));
This sums 1e5 numbers in 0.28s on my machine and 1e6 in 1.87s. (Why is it faster? Presumably because it only writes one line of output, and writing to STDOUT is apparently more expensive than parsing ints and doing addition.)
Comments
Check process.stdin
Depending on what you want to do you can even use pipe() to attach a Writable stream to the process.stdin.
readlineseems to the wrong tool for the job.