Is there any directory walker in ts / js using an async iterator?

Question

I found plenty of walkers on npm but none is using an asynchronous iterator. Most of them are either using a callback or a promise leading to memory leaks on huge directories.

Is there any recent library using the following pattern:

async function* walk(dirPath) {
    // some magic…
    yield filePath;
}

To then use it like:

for await (const filePath of walk('/dir/path')) {
    console.log('file path', filePath);
}

Library recommendations are, as you should know by now, off topic here. — jonrsharpe
– jonrsharpe, Commented May 24, 2019 at 20:16

Flavien Volken · Accepted Answer · 2019-05-25 20:43:04Z

1

Okay, I simply made this walker using the synchronous readdir, it is very fast and memory efficient, I listed 2.5 millions of entries in around 3 minutes without any memory leak.

import path from 'path';
import fs, {Dirent} from 'fs';

function* walk(path:string):IterableIterator<string> {

    const entries:Dirent[] = fs.readdirSync(path, {withFileTypes: true});

    for (const entry of entries) {
        const entryPath:() => string = () => `${path}/${entry.name}`;

        if (entry.isFile()) {
            yield entryPath();
        }

        if (entry.isDirectory()) {
            yield* walk(entryPath());
        }
    }
}

Example of usage:

for (const path of walk(directoryPath)) {
    console.log(path);
}

answered May 25, 2019 at 20:43

Flavien Volken

21.5k14 gold badges114 silver badges147 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Is there any directory walker in ts / js using an async iterator?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related