0

I am going to implement sorting using many machines (distributed sorting). I will code it with Scala. Because I do not have much experience with network programming, I need you guys help me to choose which library and method to make the communication between machines:

1/ How many machines can communicate using Message (this will be implement like case class in Scala)? Can I use ServerSocketChannel? Or should I build a TCP server for each machine? Do I need to serialize and de-serialize my object Message? If needed, can I use protobuf for doing that?

2/ How can I send data via network? Should I use java.io or java.nio?

I need to build my project using as less dependencies as possible. Thank a lot.

Improve this question
1
  • The answer to most of these is "it depends" - on what you/r'e sorting, where the data is held, the volumes compared to the number of machines and so on. Commented Oct 2, 2016 at 7:50

1 Answer 1

Reset to default
2

I would start with the simplest high level messaging library and since you have chosen Scala I would start with Akka which was designed with Scala in mind.

How many machines can communicate using Message

You can communicate with every machine you can connect to e.g. the entire internet.

If needed, can I use protobuf for doing that?

I suspect you don't need to, but yes.

How can I send data via network?

I would use a messaging library

Should I use java.io or java.nio?

The messaging library is likely to use these, but I wouldn't assume you need to use them directly.

I am going to implement sorting using many machines (distributed sorting).

There are libraries designed for Scala which do this already. I would use one of those e.g. Spark or Hadoop.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I am not encouraged to use Hadoop, Spark, Akka. Any standard library?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.