2

Im interested in programming a Project which distributes a certain computation on large files throughout several computers. The need for distributed computing arises from the crashy and unstable nature of the software I'm using to do the actual computing - so it might crash on some computers but others will surely do the job. The ideas I have so far Include: -Using several servers, each pulling a task from a master server whenever its possible -Using VMwares -Using a load-balancing Cluster

What is more suited for the job? Any other ideas I should be aware of?

Also, If you can recommend any reliable distributed computing C# framework, it will be helpful.

Improve this question
2
  • possible duplicate of stackoverflow.com/questions/1755180/… Commented Apr 22, 2010 at 17:59
  • Im not interested in CPU intensive computations but rather in a mechanism that will ensure I get all of my computations done. Commented Apr 25, 2010 at 7:50

3 Answers 3

Reset to default
1

haven't used any of these myself (yet), but I bookmarked this question a little while ago. some good suggestions there.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Have you looked at Hadoop MapReduce? It's an open-source implementation of Google's MapReduce framework. Though it's Java and not C#, it sounds like it could be perfectly suited to your scenario; the master server automatically handles load balancing and fault-tolerance in a distributed environment.

Improve this answer

1 Comment

Map reduce is definitely the way to go for this. Implementations may be a bit harder to find on the .net framework though.
0

I would check out Appistry CloudIQ Platform. It links together multiple machines into a single computing framework identified by a unified address. Your client simply submits jobs to the unified address, and the framework distributes jobs to individual machines. It also monitors task execution, and can automatically restart failed jobs. So if your application is prone to crashing, this framework could be ideal. Rather than submitting the same job to multiple machines (and wasting CPU) to cover the failure case, just submit it once, and let the framework handle restarting the jobs that actually fail. I would consider it ideal for your reliability concerns.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.