In Node.js how can I programmatically retrieve many hash's from a redis db using a set as indices

Question

I have a whole bunch of fields for each user in my redis database, and I want to be able to retrieve all their records and display them. The way I do it, is store a set of all userids, When I want all their records, I recursively iterate the set grabbing their records using the userids in the set and adding them to a global array, then finally returning this global array. Anyway I don't particularly like this method and would like to hear some suggestions of alternatives, I feel there must be better functionality in node.js or redis for this very problem. Maybe there is a way to do away with using the set entirely, but looking around I couldn't see anything obvious.

This is an example of my psuedoish (pretty complete) node.js code, note the set size is not a problem as it will rarely be > 15.

Register Function:

var register = function(username, passwordhash, email){

    // Get new ID by incrementing idcounter
    redis.incr('db:users:idcounter', function(err, userid){

        // Setup user hash with user information, using new userid as key
        redis.hmset('db:user:'+userid, {
                'username':username,
                'passwordhash':passwordhash,
                'email':email
            },function(err, reply){

                 // Add userid to complete list of all users
                 redis.sadd('db:users:all', userid);

            }
        });
    });
}

Records retrieval function: var getRecords = function(fcallback){

    // Grab a list of all the id's
    redis.smembers('db:users:all', function(err, allusersids){

        // Empty the returned (global) array
        completeArray = [];

        // Start the recursive function, on the allusersids Array.
        recursive_getNextUserHash(allusersids, fcallback);
    });  
}

Recursive function used to retrieve individual records:

// Global complete Array (so recursive function has access)
var completeArray = [];

// recursive method for filling up our completeArray
var recursive_getNextUserHash = function(userArray, callback){

    // If userArray==0 this means we have cycled entire list, 
    // call the callback, and pass it the completeArray which 
    // is now full of our usernames + emails

    if(userArray.length==0){
        callback.apply(this, [completeArray]);
        return;
    }

    // If still more items, start by popping the next user
    var userid = userArray.pop();

    // grab this users information
    redis.hvals('db:user:'+userid, function(err, fields){

        // Add users information to global array
        completeArray.push({username:fields[0],email:fields[2]});

        // Now move on to the next user
        recursive_getNextUserHash(userArray, callback);
    });

}

Use would be something like this:

register('bob', 'ASDADSFASDSA', '[email protected]');
register('bill', 'DDDASDADSAD', '[email protected]');
getRecords(function(records){
    for(var i=0;i<records.length;i++){
         console.log("u:"+records[i]['username']+',@:'+records[i]['email']);
    }
});

Summary: What is a good way to retrieve many fields of Hash's using node.js and redis? After writing this question, I started to wonder if this is just the way you do it in redis, you make many roundtrips, regardless if this is the case, there must be a way to avoid the horrible recursion!

Finbarr · Accepted Answer · 2012-03-10 06:27:08Z

4

Assuming you are using https://github.com/mranney/node_redis - have a look at Multi and Exec. You can send all of your commands in a single request and wait for all the responses at once. No need for recursion.

answered Mar 10, 2012 at 6:27

Finbarr

32.3k13 gold badges67 silver badges94 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Josh Mc Over a year ago

Yep sure am, I have looked at that, and didn't quite understand the syntax. Like I understand that you set a bunch of commands and they get executed one after another, however I don't understand how you could use the result of a command as the key for a following command, would you be able to point me in the right direction, like how to get results from a bunch of hashes using results from another command getting ids from a set?

Josh Mc · Accepted Answer · 2012-03-18 21:16:32Z

3

For anyone else having a similar question, here is the syntax I ended up using:

redis.smembers('db:users:all', function(err, reply){
     var multi = redisClient.multi();
     for(var i=0;i<reply.length;i++){
             multi.hmget('db:user:'+reply[i], ['username', 'email']);
     }
     multi.exec(function(err, replies){
             for(var j=0;j<replies.length;j++){
                  console.log("-->"+replies[j]);
             }
     });
});

edited Mar 18, 2012 at 21:16

answered Mar 10, 2012 at 7:57

Josh Mc

10.3k8 gold badges57 silver badges68 bronze badges

1 Comment

Linus Thiel Over a year ago

You may also want to consider hgetall, which gives you an object in return.

Collectives™ on Stack Overflow

In Node.js how can I programmatically retrieve many hash's from a redis db using a set as indices

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related