1

I have a REST web service which is consuming too many CPU resources when the number of requests gets too high.

This is believed to be caused by a while() loop in the response generation, which normally takes a few milliseconds to complete, however under some circumstances it can take a few seconds.

The fix for this appears to be to use wait() and notify() according to this however I don't understand why this will reduce CPU usage?

Would this new thread be handled outside the web service, thereby freeing it up to handle more requests? Can someone please explain?

Thanks!

Jon.

Edit:

I may have found my own answer here

It seems that my code of result = get() constantly polls until there is a response, which consumes more CPU resources. By placing this in a thread, less resources are consumed.

Is this a correct understanding?

Improve this question
3
  • Would be much easier if we could see some code... Commented Mar 9, 2012 at 11:30
  • Hard to say since we don't have your code to verify. If in your case, you have two thread, one that will produce something and the other which polls constantly to see if the producer produced something, then yes, you should avoid that and you can organize the Threads execution with the use of wait and notify. Commented Mar 9, 2012 at 11:30
  • Apologies for the lack of code, it's a substantial calculation and I don't know where the problem area is. I think Stephen has uncovered the issue below though. Commented Mar 9, 2012 at 11:55

2 Answers 2

Reset to default
5

Is this a correct understanding?

A loop that continually polls something until there is a response is very wasteful. If you add a sleep between each poll, you reduce the CPU usage, but at the cost of reduced responsiveness for individual requests ... compared to what is achievable if you do it the right way.

Without knowing exactly what you are doing (what you are polling, and why) it is a bit difficult to say what the best solution is. But here are a couple of possible scenarios:

  • If your web service is waiting for a response from an external service, then the simple solution is to just do a blocking read, and configure your web server with more worker threads.

  • On the other hand, if your web service is waiting for a computation to complete, a new thread and wait / notify ... or one of the higher level synchronization classes ... may be the answer.

  • If you need to handle a really large number of these blocking requests in parallel, that is going to require a lot of threads and hence a lot of memory, and other things. In that case you need to consider an web container that breaks the one-thread-per-request constraint. The latest version of the Servlet spec allows this, as do some of the alternative (non-Servlet) architectures.

FOLLOW UP

... I think the issue is your point 2, that the service is simply just waiting for the computation. So, by simply threading this computation will free up resources in the service?

If what you are describing is true, then running the computation in a different thread won't make it go much quicker. In fact, it could make it go slower.

The ultimate bottleneck is going to be CPU capacity, disc bandwidth and / or network bandwidth. Multi-threading is only going to make an individual request go faster if you can effectively / efficiently use 2 or more processors on the same request at the same time. It will only make your throughput better to the extent that it allows requests to run while others are waiting for external events; e.g. network responses to arrive, or file read/write operations to complete.

What I think you actually need to do is to figure out why the computation is taking so long and try and fix that:

  • Are your database queries inefficient?
  • Are fetching result-sets that are too large?
  • Do you have a problem with your schemas?
  • Poor choice of indexes?
  • Or are you simply trying to do too much on a machine that is too small using the wrong kind of database?

There are various techniques for measuring performance of an application service and a database to determine where the bottlenecks are. (Start with a Google search ....)

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Sorry for the lack of code - it's actually a very large calculation, so I really can't post all of it here and I'm not sure of the problematic area. Looking through it, there is not really any polling happening that can be removed. The only loops are some for() and while(sqlResultSet.net()) loops - so I don;t think this is the problem? I think the issue is your point 2, that the service is simply just waiting for the computation. So, by simply threading this computation will free up resources in the service? Same as you need to keep background processing free from the GUI?
Server specs should not be the issue. 4 processors, 16gb ram. There is a large dataset being loaded for each request, it's not slow doing this, but there is a large number of requests happening at once up to 400 per minute. Ideally this could be loaded once initially and kept in memory for use by each request - but I do not know how I can do this?
Is it feasible to move the bulk of the work into the query, so that you either pull back just the result or at least a smaller data set for each request?
0

Firstly, stop trying to guess why it's consuming so much CPU. Put some instrumentation in place and find out where the bottle-neck really is. Once you know where the bottle-neck is, try to to understand the root cause. 5 whys is a good technique to use when doing this.

Once you know the root cause, determine if it's solvable. e.g. if the algorithm is inefficient and there is an efficient algorithm is available then refactor the code to use the efficient algorithm. Don't forget that the root cause might be the server is under-powered and needs a bigger CPU.

Secondly, consider using a http cache to reduce the load on your server. How frequently does the data requested change and how fresh do you responses need to be? Use this to calculate a max-age; the length of time a HTTP cache could keep a response. It might be 1 min, a day, a week, etc. It really depends on what you are serving up. Once you have a max-age, have a look at the requests coming in. How many are for the same URI (or could be for the same URI if you tweeked it) within the max-age you've picked. If all the request URIs within a max-age period are unique then caching won't help you. If you're getting on average 2 requests for each URI in a max-age period, then a HTTP cache will half the load on you server. If it's 10 requests per URI, then a HTTP cache will reduce the server load by 90%.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.