6

I am trying to write a program in C# that will connect to around 400 computers and retrieve some information, lets say it retrieves the list of web services running on each computer. I am assuming I need a well threaded application to be able to retrieve info from such a huge number of servers really quick. I am pretty blank on how to start working on this, can you guys give me a head start as to how to begin!

Thanks!

Improve this question
8
  • 1
    Why are you assuming that you need multiple threads? Why not access them one at a time? Commented Jun 16, 2011 at 10:26
  • Are you going to write client/server application? Commented Jun 16, 2011 at 10:26
  • More than an assumption you can say it is a rule that has been laid on me, I have been instructed to multithread the application. Correct me if I am wrong, if i made blocking calls, few servers might take a long time to return replies but there might be faster ones that could return the result in the mean time. Hope this makes sense. Commented Jun 16, 2011 at 10:29
  • @zabulus, only client side, I have already written the console app for the single server model. Now need to expand from 1 to a huge number of systems! Commented Jun 16, 2011 at 10:30
  • Do you want the responses to be at nearly the same time, or does it make sense to simply roll through the list computers? Commented Jun 16, 2011 at 10:30

6 Answers 6

Reset to default
5

I see no reason why you should use threading in your main logic. Use asynchronous APIs and schedule their callback to the main thread. That way you get the benefits of asynchrony, but without most of the difficulty related to threading.

You'll only need multithreading in your logic code if the work you need to do on the data is that expensive. And even then you usually can get aways with parallelizing using side effect free functions.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Thanks for that idea, I will definitely go ahead with the approach suggested by you. Will get back as soon as my results come out.
To say that using the async APIs is not using threading is somewhat of a simplification. Agreed that explicit threading is abstracted away, but given that the callbacks from async APIs are likely to occur on different threads, it still presents a lot of the same issues that explicit multithreading incurs.
1) There are non blocking APIs that don't use threading at all 2) Even if you use an API that has callbacks on a different thread, you can schedule it to the main thread. For example using the Post method on an appropriate synchronization context.
Hey i've a small doubt, hope you see this soon because i'll be starting work on this real soon. So is there a method to know that the data returned from an async call is from which source? As in my question i have multiple servers and when data is returned will i be able to distinguish from which server the info has come?
I've never used there async APIs. But usually when you subscribe an event somewhere you can either pass in a userdata objects, or use the this of the delegate as userdata.
2

Take a look at the Task Parallel Library.

Speficically Data Parallelism.

You could also use PLINQ if you wanted.

Improve this answer

Comments

1

You should also execute the threads parallely on a multi-core CPU to enhance performance.

My favourite references on the topic are given below -

http://www.albahari.com/threading/

http://www.codeproject.com/KB/Parallel_Programming/NET4ParallelIntro.aspx

Improve this answer

3 Comments

What are the performance issues when the OP is talking about consuming web services?
He is looking for the best possible way to get the job done. The operations he want to do seem to be independent of each other. Dont you think executing them parallely makes sense?
+1 for Joe Albahari's threading pages. A really solid primer.
0

Where and how do you get the list of those 400 servers to query?

how often do you need to do this?

you could use a windows service or schedule a task which invoke your software and in it you could do a foreach element in the server list and start a call to such server in a different thread using thread queue/pool, but there is a maximum so you won't start 400 threads all together anyway.

describe a bit better your solution and we see what you can do :)

Improve this answer

3 Comments

the server list is part of a SQL Server db that already exists. It's like a standalone application so no need to schedule it, i'll probably run it once a day or something based on the need. It is more like a consistency checking tool that I am aiming at.
Ok, still I would not put the logic in the UI but in a class library so you could unit test, use from a windows front end or from web pages and eventually in the future also schedule with little effort. The other answers you have got are fine in my opinion.
I did not get your point about UI being separated from Logic, i would like to create a visual report pointing to the spot of inconsistency. I think i need to rephrase my question.. Here goes, in simplified words, I have a list of 10 servers that make a cluster, now the main feature of this is that each server has a local copy of a file in this case a .svc or .asmx file, and all the data is same. Now when I run my tool i need to be able to fetch info of these services and check if any of the servers has aberrant data. If yes, then i need to visually show the point of error.
0

Take a look at this library: Task Parallel Library. You can make efficient use of your system resources and manage your work easier than managing your threads directly.

Improve this answer

Comments

0

There might be considerable impact on the server side when you start query all 400 computers. But you can take a look at Parallel LINQ (PLINQ), where you can limit the degree of parallelism.

You can also use thread pooling for this matter, e.g. a Task class.

Createing manual threads may not be a good idea, as they are not highly reusable and take quite a lot of memory/CPU to be created

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.