NodeJs : Code sometimes work - Async.map issue

Question

I have a config json file as below.

"constraints": {
    "input": "input",
    "output": "output"
  }

I am trying to read this file and create input and output directory with child directories.

var fs = require("fs");
var async = require("async");
var node_xj = require("xls-to-json");
var cf= JSON.parse(fs.readFileSync("config.json", 'utf8'));

// Declare variables
var files = [];    

function readAsync(file, callback) {
    node_xj(file, callback);        
} 

function create(currentDirPath, outputDirPath, callback) {  
// some code creating directories
}    

create(cf.lang.input, cf.lang.output, function(stat, config) {
    files.push(config); 
});  

async.map(files, readAsync, function(err, results) {
     if(err) throw err;
});

The code works fine, but its sometimes does not. Let me walk through the code. I am loading some modules.

var fs = require("fs");
var async = require("async");
var node_xj = require("xls-to-json");

I am loading the config file.

var cf= JSON.parse(fs.readFileSync("config.json", 'utf8'));

Then i am passing the cf file to create function which after its operation returns me an an object which i push it into an array.

var files = [];    
function readAsync(file, callback) {
    node_xj(file, callback);        
} 

function create(input, output, callback) {  
   // some code creating directories and object
}    

create(cf.lang.input, cf.lang.output, function(stat, config) {
    files.push(config); 
});  

async.map(files, readAsync, function(err, results) {
     if(err) throw err;
});

Then i am passing the files array to my async.map function which passes it to readAsync function for another operation.

Question:

Can anyone tell me whether the way i have written the code flow makes if flaw sometimes.
Is there a better way of writing the same code logic flow.
Should i use async.map to iterate files and then pass it to readAsync

Rob Raisch · Accepted Answer · 2015-04-16 22:04:16Z

1

If you're depending on asynchronous operations to occur before other operations, you have to treat them asynchronously! Your code as described seems to create some directories (which will take measurable time) and immediately attempts to use them, which isn't kosher.

You might consider something like:

async.series([
  function(){
    // create your directories here
  },
  function(){
    async.map(files,...)
  }
]);

which will guarantee that the resources needed by your map exist before the map is called.

answered Apr 16, 2015 at 22:04

Rob Raisch

17.5k4 gold badges51 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Shane Over a year ago

Thanks, in this series... how will i pass my config param to my function? I am not not quite getting it... is it possible can you give a little more code on the same converting mine? Thanks again.

Rob Raisch Over a year ago

Sure, if you could provide a clear explanation of what you're trying to accomplish. For example, you have a function readAsync() which doesn't read anything. Rather is converts an xsl file into JSON but doesn't indicate where that file came from. ;) IF you can provide a list of steps of what you need to accomplish, I'd be happy to show you how.

Collectives™ on Stack Overflow

NodeJs : Code sometimes work - Async.map issue

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related