2

I've been reading about node.js recently (like many others). I find interesting for some use cases, but am a bit struggling to understand the inner workings, specifically the interaction between closure functions and the process flow of the code.

Let's say I have a function which accepts a key-value array. The function must check that the values follow certain data-quality guidelines (for example some keys must have a value, other keys must have numbers as values etc) before storing the data somewhere (for the purpose of this question let's assume data validation has to be done in the application itself).

In "regular" developments models I'd write something like this:

resultName = validateName(data.name)
resultAddress = validateAddress(data.address)
resultID = validateID(data.id)

if (resultName && resultAddress && resultID) {
    store(data)
else {
    sendErrorToUser(data)
}

Get the results of the validations, and either explain the error(s) to the user or store data and return some kind of confirmation. The flow is very clear.

The way I understand node.js, the way to do this would be to delegate the validations to a different function (to avoid waiting for each validation to finish), and supply two callback functions to the functions which validate the chunks of data: * a callback to call when validation is successful * a callback to call when validation fails

It's easy to now return to the user with a "please wait" message, but I have to wait for all validations to clear (or fail) before storing the data or explaining the problem to the user. As a simple way to figure out if all the validations are done I thought of using a variable that counts the number of functions that called the callback, and emitting a "validation complete" event to store the validated data (or get back to the user with any errors). Or, alternatively, emit an event after each validation is complete and in that event's code check if all validations are complete before emitting the "store" / "error" events.

My question is -- am I approaching this correctly? Or is there a more suitable way to do these kinds of things with node.js (or similar event-based systems).

Thank you! Alon

Improve this question

4 Answers 4

Reset to default
2

Are your validations asynchronous? If they are not you can use the code you posted, the "regular" one.

If the validations are asynchronous (checking uniqueness of an email for instance), you need to provide callbacks:

var validateUniqueEmail = function (data, callback) {
  db.find({email: data.email}, function (err, result) {
    callback(err, result === null);
  })
};

var validateAndStore = function (data, callback) {
  asyncValidation(data, function (err, is_valid) {
    if (err) {
      callback(err, null);
    } else if (!is_valid) {
      callback('Email is not unique', null);
    } else {
      db.store(data, callback);
    }
  });
}

The code above can be simplified a lot by using some validator or ORM modules already existing

example: mongolia validator module.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

so how would the code calling the validations and acting on them look like with multiple async validations? will the validations actually happen simultaneously? if so do they all share the same callback function? I apologize for the flood of questions, I'm struggling to get my head around this.
You can run all the validations in parallel and then callback with an array of errors found.
1

Let's go. Basically, what you want to do is something along the lines of : 

var validate(data, cb){
  var allOk = true;
  for(var key in data){
    allOk = allOk && validate[key](data.key); // validator depends on the key
  }
  if (allOk) cb(null, data); else cb(new Error "bleh");

}

This could be done the following way (note how we pass the failed keys as the first (error) argument to the callback):

var validate(data, cb){
  var status = {true:[], false:[]},
      total  = Object.keys(data).length,
      done   = 0;

  for (var key in data)
    (function(key){
      validate[key](data[key], function(ok){
        status[ok].push(key);
        if (++done == total){
          status[false].length ? cb(status[false]) : cb(null);
        }            
      });
    })(key);
}

Which you can use this way : 

validate(data, function(failures){
  if (failures){
    // tell the user the input does not validate on the keys stored in failures
  } else {
    // all ok
    store(data);
  }
});
Improve this answer

2 Comments

I assume the second call to 'validate[key]' should be a different function that actually does some kind of validations?
That's it. It allows for more extensions in the future than hardcoding your validators. if someday you need to add a field, for example email, you'd just have to add validate.email without modifying the actual source here. the only requirement is that the signature of the validator is function (err, callback).
0

Correct me if I'm wrong, but I think what you're asking is how to handle the response from multiple asynchronous calls.

Here's how I do it (using your validation example):

var result = {};

function isAllDataAvailable() {
    return result.name    !== undefined
        && result.address !== undefined
        && result.id      !== undefined;
}

function callback(error) {
    if (error) {
        showError(error);

        // terminate here (?)

        return;
    }

    if (isAllDataAvailable()) {
        showOutput();
    }
}

validateName(data, callback);
validateAddress(data, callback);
validateEmail(data, callback);

The key here is the result object, which starts out as empty. As each field gets validated, it gets added to the result object (by the validation functions, which I've left out in the above snippet). I've used a single callback method, but you could have multiple, say callbackName, callbackAddress, etc. The validation results are processed only if and when the result object has been fully populated, which is checked in isAllDataAvailable.

Hope this helps.

Improve this answer

5 Comments

So if I understand correctly: the result object is defined in the context of the function calling the validation functions, and as such it becomes a global variable available to any work spawned from that function (the validation function); all three validateXXXX functions will be called one after the other, but without waiting for each function to complete (does that happen using threads behind the scenes?); the result object is thread-safe by default and hence can be shared amongst the different validations. Am I getting this right?
Let's say the above code is within a validate function. Then each call to validate would have its own copy of a result object, yes. You could pass result to validateName, validateAddress, etc., as the third parameter. The important thing here is that the callback always checks result through isAllDataAvailable before printing the output.
Also, as to whether validateName returns immediately, that depends on what it does. From your question it appears that you have some form of asynchronous validation (which does return immediately but performs the task in the background - using threads or whatever node.js does).
I guess my initial question was mostly about "...which does return immediately but performs the task in the background - using threads or whatever node.js does" -- how does node.js handle this? will it be opening threads for the background jobs?
There are ways of doing asynchronous I/O depending on the underlying system (Linux, Windows, etc.). Search Google for "asynchronous io" and you'll find a few links. I'm not sure exactly what Node.js does.
0

Consider using: https://github.com/asaf/nodejs-model It will make your life much easier when dealing with validators.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.