TypeScript: async generator

Question

I'd like to have a function like this:

export async function* iterateDir(dir: string) {
    let list = await fs.readdir(dir); // fs-promise implementation of readdir
    for (let file of list) {
        yield file;
    }
}

Which I would use like:

for (let file in iterateDir(dir)) {
    processFile(file);
}

This doesn't work because a function cannot be both async and a generator.

How would I structure the code to achieve the same?

If I change the await fs.readdir to callbacks, I assume the outer for..of loop would not wait.
If I get rid of the generator and the directory is huge, iterateDir() will be slow.

For reference: async generator function proposal

If your async work is done before your generator work, then you can just make them different functions. — Stephen Cleary
– Stephen Cleary, Commented Nov 3, 2016 at 16:18
The point is that the generator itself needs to be async (it awaits filesystem operations). — Borek Bernard
– Borek Bernard, Commented Nov 3, 2016 at 16:21
Generators cannot be asynchronous (yet). The code you posted doesn't need to be an asynchronous generator; it can be split into an asynchronous part and a generator part. — Stephen Cleary
– Stephen Cleary, Commented Nov 3, 2016 at 16:31
Well, in my specific case, iterateDir would be recursive. For every file, there would be a detection whether it's a directory or not, and if it is, call iterateDir recursively. In which case I don't know how to split the async and generation functionality. Which is the point of the question.. — Borek Bernard
– Borek Bernard, Commented Nov 3, 2016 at 16:41

Community · Accepted Answer · 2017-05-23 12:34:21Z

38

This is supported in TypeScript 2.3 - tracked issue

It introduces a few new types, notably:

interface AsyncIterable<T> {
    [Symbol.asyncIterator](): AsyncIterator<T>;
}

but most importantly it also introduces for await ... of

for await (const line of readLines(filePath)) {
  console.log(line);
}

where

async function* readLines(path) {
   //await and yield ...
}

Be aware that if you want to try this you will need to configure typescript to let it know you have run-time support (add "esnext.asynciterable" to lib list) you will probably need to polyfill Symbol.asyncIterator. see TS2318: Cannot find global type 'AsyncIterableIterator' - async generator

edited May 23, 2017 at 12:34

CommunityBot

11 silver badge

answered Feb 16, 2017 at 10:00

Meirion Hughes

26.7k13 gold badges80 silver badges130 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jarda Snajdr · Accepted Answer · 2016-11-12 20:44:10Z

-3

Stream of future values is called Observable. So the most natural thing is to use a library like RxJS or most.js. If you don't want to introduce a new complex library only to use it once, use the good old approach with callbacks:

const path = require("path");
const fs = require("mz/fs");

async function listDirectory(dirName, cb, level = 0) {
  for (let fileName of await fs.readdir(dirName)) {
    let absName = path.resolve(dirName, fileName);
    let isDirectory = (await fs.stat(absName)).isDirectory();
    cb({ level, fileName });
    if (isDirectory) {
      await listDirectory(absName, cb, level + 1);
    }
  }
}

listDirectory(".", ({ level, fileName }) => {
  console.log(" ".repeat(level) + fileName);
});

If you try to convert the callback to some better abstraction, you will rediscover RxJS sooner or later. It's similar to converting one-shot callbacks to promises.

answered Nov 12, 2016 at 20:44

Jarda Snajdr

6388 silver badges10 bronze badges

2 Comments

Coderer Over a year ago

The difference is that iterables are "pull" (caller requests next value) while Observables are "push" (called function determines when a new value is available and notifies the caller). In the OP's example, they really want a Promise (push) that resolves with a generator function (pull).

Gherman Over a year ago

I was actually trying to use Observable and failed to write code that would be sensibly simple. The code I got was absurdly complex and I gave up on Observables. Then I was trying to find code examples and found none pretty much.

Collectives™ on Stack Overflow

TypeScript: async generator

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related