I'm writing a Cloud Function that reads several lines from a .csv file and sends one HTTP request for each line, but I'm getting the error above.
This error has been asked before (here, here and here to list a few), but in all of these cases, the issue was that the function didn't return a promise that would resolve only after every asynchronous operation.
From my understanding of how async works, my code doesn't seem to have this issue. Here's the gist of it, removing some specific details:
const request = require('request-promise-native');
const storage = require('@google-cloud/storage')();
exports.sendData = async (data, context) => {
const [file] = await storage.bucket(data.bucket).file(data.name).download();
const [header, ...table] = file.toString('utf8').split('\n').map(row => row.split(','));
const rows = table.map(
row => header.reduce((obj, col, i) => {
obj[col] = row[i];
return obj;
}, {})
);
const paramList = rows.map(({ csv_field1, csv_field2 }) => ({
req_field1: csv_field1,
req_field2: csv_field2,
// etc.
}));
let errors = 0;
console.log(`Sending ${paramList.length} rows...`);
await Promise.all(paramList.map(async (param, i) => {
try {
await request.get({ url: "", qs: param }); // Endpoint removed
if ((i + 1) % 10000 == 0) {
console.log(`${i + 1} rows ${paramList.length} sent.`);
}
} catch (e) {
console.log(`Request failed for row ${i + 1} .\n\n${e}`);
errors++;
}
}));
console.log(`Done.`);
if (errors > 0) {
console.log(`Total errors: ${errors}`);
}
};
The above code works just fine for a couple of lines (10), but the error begins showing up when trying with bigger numbers (1000). The "Done." message also didn't appear in the logs, so it doesn't seem that the function is reaching the end.
Any clues?
