Close a Redis connection based on Node.js Event loop

Question

Issue/ How I ran into it

I'm writing a batch processing Lambda function in Node.JS that makes calls to Redis in two different places. Some batch items may never reach the second Redis call. This is all happening asynchronously, so if i close the connection as soon as the batch queue is empty, any future Redis calls would fail. How do I close the connection?

What i've tried

process.on('beforeExit', callback) -- Doesn't get called as the event loop still contains the Redis connection
client.unref() -- Closes connection if no commands are pending. Doesn't handle future calls.
client.on('idle', callback) -- Works but is deprecated and may still miss future calls

What I'm currently doing

Once the batch queue is empty, I call:

intervalId = setInterval(closeRedis, 1000);

I close the Redis connection and clear the interval in the callback after a timeout:

function closeRedis() {
    redis.client('list', (err, result) => {
        var idle = parseClientList(result, 'idle');

        if (idle > timeout) {
            redis.quit();
            clearInterval(intervalId);
        }
    });
}

This approach mostly works, but if just checking for a timeout, there is still a chance that other processes are going on and a Redis call may be made in the future. I'd like to close the connection when there's only an idle connection remaining in the event loop. Is there a way to do this?

brandav · Accepted Answer · 2018-01-15 18:54:29Z

2

I ended up using process._getActiveHandles(). Once the batch queue is empty, I set an interval to check every half a second if only the minimum processes remain. If so I unref the redisClient.

redisIntervalId = setInterval(closeRedis, 500);

// close redis client connection if it's the last required process
function closeRedis() {
    // 2 core processes plus Redis and interval
    var minimumProcesses = 4;
    if (process._getActiveHandles().length > minimumProcesses)
        return;

    clearInterval(redisIntervalId);
    redisClient.unref();
}

The advantage of this approach is that I can be sure Redis client will not close the connection while other important processes are running. I can also be sure that the client won't keep the event loop alive after all the important processes have been completed.

The downside is _getActiveHandles() is an undocumented node function so it may get changed or removed later. Also, unref() is experimental and doesn't consider some Redis commands when closing the connection.

answered Jan 15, 2018 at 18:54

brandav

7955 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

demian85 Over a year ago

If you are developing a lambda function, you can just set context.callbackwaitsforemptyeventloop = false to end function execution without waiting for the connection to close.

Collectives™ on Stack Overflow

Close a Redis connection based on Node.js Event loop

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related