5

I was wondering if somebody could point me to a simple equivalent of python's multiprocessing module in java.

I have a simple parallel processing scenario (where no 2 processes interact): Take a data set and split it into 12 and apply a java method to the 12 datasets, collect results and join them in a list of some sort with the same ordering.

Java being a "pro" language appears to have multiple libraries and methods - anyone who can help this java newbie get started?

I would like to do this with minimal of coding - as i said my requirement is pretty straightforward.

Update: how to do multiprocessing in java, and what speed gains to expect?

This seems to indicate threads is the way to go. I expect I have no choice but wade into a bunch of locks (pun unintended) and wait for my ship to sail. Simple examples are welcome nevertheless.

Improve this question
4
  • You mean like docs.oracle.com/javase/7/docs/api/java/util/concurrent/… ? Commented Aug 19, 2013 at 23:48
  • Possibly - With the multiprocessing module i don't have to think about threads and stuff. I simply map my function across the chunks of data in a list. The concurrent util here seems more complex at least to me. Perhaps there is an example for a simple application that will get me started. Commented Aug 19, 2013 at 23:53
  • stackoverflow.com/questions/4208208/… this was helpful too - but if there is anything simpler out there would appreciate it. Commented Aug 19, 2013 at 23:58
  • ricardozuasti.com/2012/… gets me partway there using the concurrent class. Commented Aug 20, 2013 at 0:07

1 Answer 1

Reset to default
4

There's no exactly-compatible class, but ExecutorService gives you everything you need to implement it.

In particular, there's no function to map a Callable over a Collection and wait on the results, but you can easily build a Collection<Callable<T>> out of a Callable<T> and Collection<T>, then just call invokeAll, which returns you a List<Future<T>>.

(If you want to emulate some of the other functions from multiprocessing.Pool, you will need to loop around submit instead and build your own collection of things to wait on. But map is simple.)

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

That is exactly what I needed - I'll test it tomorrow and may get back with some questions.
@pythOnometrist: If you want to do anything non-trivial, I'd suggest learning how executors and futures work, instead of trying to build the Python multiprocessing API on top of them. You might want to play with Python's own concurrent.futures library (or the backport if you're on 2.x) to get the hang of the ideas first. A lot of code is actually simpler that way anyway. But also, if you have to use Java, you'll be a lot happier writing idiomatic Java than trying to use Java as a very bad Python…
I do not understand. How is this multiprocessing? Isn't ExecutorService made for threads only?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.