Rate limit API queries in node.js

Question

I am attempting to rate limit queries to the Battle.net API up to their limits of 100 calls/second, 36,000/hour.

My current code looks like this:

var async = require ('async');
var bnet = require ('battlenet-api');

async.eachSeries(regions, function(region, callback) {
    bnet.wow.realmStatusAsync({origin: region}).spread(/*...snip...*/)
});

What I'd need here is to make sure that no calls to battlenet-api run up to the limit, and if so, queue them.

I have looked at timequeue.js, limiter and async's own capabilities and none seems to provide what I am looking for.

Does someone know how to achieve this?

So, you want to implement your own limiter to make sure battle.net's rate limiting isn't hit from your service? — user3117575
– user3117575, Commented Aug 23, 2015 at 15:20
Not exactly, I'd like to know if there's any solution out there to achieve it. — amyspark
– amyspark, Commented Aug 23, 2015 at 15:43
Questions asking for a third party library solution are considered off-topic at StackOverflow. — jfriend00
– jfriend00, Commented Aug 23, 2015 at 16:17

dm03514 · Accepted Answer · 2017-03-09 18:37:57Z

2

A couple possibilities are:

Wait until you get throttled and backoff your requests
Keep track of your limits and used limits inside of your application.

While i think both cases need to be handled, the second one is more valuable, because it will let your application know at any time how many requests have been made for a given timeframe.

You could focus on implementing this on the hour level and letting the backoff/retry catch the cases of going over the 100 req/s timelimit.

To implement this, your application would have to keep track of how many requests are available for the given time period. Before each request the request count has to be checked to see if there are available requests, if there are the API request can be made and if not then no request will be made and your application will throttle itself. Additionally, the limits need to be reset when appropriate.

There are quite a few articles on using redis to accomplish this, a quick google search should uncover them

edited Mar 9, 2017 at 18:37

answered Aug 23, 2015 at 16:22

dm03514

56.2k18 gold badges117 silver badges147 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Kilizo Over a year ago

Please provide links to the articles you mentioned.

Collectives™ on Stack Overflow

Rate limit API queries in node.js

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related