2

I am currently writing a small application to load test a website and am having a few problems.

List<string> pageUrls = new List<string();
// NOT SHOWN ... populate the pageUrls with thousands of links

var parallelOptions = new System.Threading.Tasks.ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = 100;
System.Threading.Tasks.Parallel.ForEach(pageUrls, parallelOptions, pageUrl =>
{
    var startedOn = DateTime.UtcNow;

    var request = System.Net.HttpWebRequest.Create(pageUrl);

    var responseTimeBefore = DateTime.UtcNow;

    try
    {
        var response = (System.Net.HttpWebResponse)request.GetResponse();
        responseCode = response.StatusCode.ToString();
        response.Close();
    }
    catch (System.Net.WebException ex)
    {
        // NOT SHOWN ... write to the error log
    }

    var responseTimeAfter = DateTime.UtcNow;

    var responseDuration = responseTimeAfter - responseTimeBefore;

    // NOT SHOWN ... write the response duration out to a file

    var endedOn = DateTime.UtcNow;

    var threadDuration = endedOn - startedOn;

    // sleep for one second
    var oneSecond = new TimeSpan(0, 0, 1);
    if (threadDuration < oneSecond)
    {
        System.Threading.Thread.Sleep(oneSecond - threadDuration);
    }
}
);

When I set the MaxDegreeOfParallelism to a low value such as 10 everything works fine, the responseDuration stays between 1 and 3 seconds. If I increase the value to 100 (as in the example) the responseDuration climbs quickly until after around 300 requests the it has reached 25 seconds (and still climbing).

I thought I may be doing something wrong so I also ran Apache jMeter with the standard web test plan setup and set the users to 100. After about 300 samples the response times had rocketed to around 40 seconds.

I'm skeptical that my server is reaching its limit. The task manager on the server shows that only 2GB of the 16GB is being used and the processor hangs around 5% effort.

  1. Could I be hitting some limit on the number of simultaneous connections on my client computer? If so, how do I change this?
  2. Am I forgetting to do something in my code? Clean-up/close connections?
  3. Could it be that my code is OK and it is in fact my server that just can't handle the traffic?

For reference my client computer that is running the code above is running Windows 7 and is on the same network as the server I am testing. The server is running Windows Server 2008 IIS 7.5 and is a dedicated 8-core 16GB RAM machine.

5
  • What about DB? I think the issue is at server side Commented Dec 10, 2012 at 18:02
  • 1
    For jmeter a typical laptop can run between 30 - 60 parallel threads without running into trouble. Jmeter needs to do bookkeeping for all those threads, that can be the source of the slowdown you see. Try running with 30 or 50 threads and see if that improves things. Inserting a timer for a few 100ms between samplers can make sure your network (card) isn't part of the percieved bottleneck. Commented Dec 10, 2012 at 21:05
  • @rsp I would like to simulate 100+ users per second hitting my server just to test the limits. Is my only option to run jMeter simultaneously on seperate client machines? Commented Dec 11, 2012 at 8:55
  • 1
    Depending on what a test with 30 threads does, you might have to run from multiple clients for 100+ concurent threads. Note that it would be possible to do 100+ hits per second with 34 threads and samples < 300ms each. (Btw, a linux server with 4 cores, 8Gb ram and fast network card should be able to run jmeter with 100 theads.) Commented Dec 11, 2012 at 9:48
  • Ok, it's all starting to become a little clearer. One of my problems in jMeter appeaers to be the Graph Results plugin, when switched with a Simple Data Writer the perceived performance stopped skyrocketing. Now getting response times of around 3 seconds with 100 users, this equates to around 30 hits a second. I am finding learning jMeter fairly hard going, I prefer the control I have in code. Commented Dec 11, 2012 at 10:27

2 Answers 2

2

MaxDegreeOfParallelism should be used only when you are trying to limit the number of cores to be used as part of your program strategy.

By default, Parallel library utilizes the most number of available threads - so setting this option to any number mostly will limit the performance depending on the environment running it.

I would suggest you to try running this code without setting this option and that should improve the performance.

ParallelOptions.MaxDegreeOfParallelism Property in MSDN - read remarks section for more information.

Sign up to request clarification or add additional context in comments.

1 Comment

I am trying to limit the number of threads as part of my program strategy. 1 thread = 1 user. If I set the MaxDegreeOfParallelism to 1 then I get just the one thread running which is exactly what I want. Ideally I would like to simulate one hundred users (or more) simultaneously hitting my server to see what its limits are. Maybe my only other option is to set it to 10 and run the program simultaneously on 10 other client machines?
1

Several suggestions:

  1. How large is your recorded Jmeter test script and did you insert some think time? The larger the test, the heavier the load.
  2. Make sure the LAN is not in use by competing traffic during test runs. Having a Gigabit ethernet switch should be mandatory.
  3. Do use 2-3 slave machines and avoid using heavy results loggers in Jmeter like tree.You were right to minimize these graphs and results.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.