Using async/await with a forEach loop

Using async/await with a forEach loop

Are there any issues with using async/await in a forEach loop? I’m trying to loop through an array of files and await on the contents of each file.
import fs from ‘fs-promise’

async function printFiles () {
const files = await getFilePaths() // Assume this works fine

files.forEach(async (file) => {
const contents = await fs.readFile(file, ‘utf8’)
console.log(contents)
})
}

printFiles()

This code does work, but could something go wrong with this? I had someone tell me that you’re not supposed to use async/await in a higher order function like this, so I just wanted to ask if there was any issue with this.

Solutions/Answers:

Solution 1:

Sure the code does work, but I’m pretty sure it doesn’t do what you expect it to do. It just fires off multiple asynchronous calls, but the printFiles function does immediately return after that.

If you want to read the files in sequence, you cannot use forEach indeed. Just use a modern for … of loop instead, in which await will work as expected:

async function printFiles () {
  const files = await getFilePaths();

  for (const file of files) {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  }
}

If you want to read the files in parallel, you cannot use forEach indeed. Each of the async callback function calls does return a promise, but you’re throwing them away instead of awaiting them. Just use map instead, and you can await the array of promises that you’ll get with Promise.all:

async function printFiles () {
  const files = await getFilePaths();

  await Promise.all(files.map(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  }));
}

Solution 2:

With ES2018, you are able to greatly simplify all of the above answers to:

async function printFiles () {
  const files = await getFilePaths()

  for await (const file of fs.readFile(file, 'utf8')) {
    console.log(contents)
  }
}

See spec: https://github.com/tc39/proposal-async-iteration


2018-09-10: This answer has been getting a lot attention recently, please see Axel Rauschmayer’s blog post for further information about asynchronous iteration: http://2ality.com/2016/10/asynchronous-iteration.html

Solution 3:

Instead of Promise.all in conjunction with Array.prototype.map (which does not guarantee the order in which the Promises are resolved), I use Array.prototype.reduce, starting with a resolved Promise:

async function printFiles () {
  const files = await getFilePaths();

  await files.reduce(async (promise, file) => {
    // This line will wait for the last async function to finish.
    // The first iteration uses an already resolved Promise
    // so, it will immediately continue.
    await promise;
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  }, Promise.resolve());
}

Solution 4:

The p-iteration module on npm implements the Array iteration methods so they can be used in a very straightforward way with async/await.

An example with your case:

const { forEach } = require('p-iteration');
const fs = require('fs-promise');

(async function printFiles () {
  const files = await getFilePaths();

  await forEach(files, async (file) => {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  });
})();

Solution 5:

Here are some forEachAsync prototypes. Note you’ll need to await them:

Array.prototype.forEachAsync = async function (fn) {
    for (let t of this) { await fn(t) }
}

Array.prototype.forEachAsyncParallel = async function (fn) {
    await Promise.all(this.map(fn));
}

Note while you may include this in your own code, you should not include this in libraries you distribute to others (to avoid polluting their globals).

Solution 6:

Both the solutions above work, however, Antonio’s does the job with less code, here is how it helped me resolve data from my database, from several different child refs and then pushing them all into an array and resolving it in a promise after all is done:

Promise.all(PacksList.map((pack)=>{
    return fireBaseRef.child(pack.folderPath).once('value',(snap)=>{
        snap.forEach( childSnap => {
            const file = childSnap.val()
            file.id = childSnap.key;
            allItems.push( file )
        })
    })
})).then(()=>store.dispatch( actions.allMockupItems(allItems)))