How to share a list of objects for parallel processing in Java?

Question

I have a list of objects that has to be shared among multiple threads.

For example, I have a 5 discrete objects and a pool of 20 threads. At first 5 of 20 threads will start working, leaving the other threads in WAIT state. 5 threads will utilize each of 5 objects at a same time.

If any of the thread is completed it should release the object so that the 6th thread can start working.

For parallel processing of threads I think I can use Executor pool. But how to share the list of objects among threads ?

Take a BlockingQueue (docs.oracle.com/javase/7/docs/api/java/util/concurrent/…) and put the objects back after you are finished — Gilfoyle
– Gilfoyle, Commented May 31, 2016 at 7:38
Is the 6th thread doing something different to the object than the 5th one? Why would you need more than 5 threads if you have only 5 objects. There's certainly no performance advantage in having 4 times the amount of threads you need. — Kayaman
– Kayaman, Commented May 31, 2016 at 8:01
Yes. All the 20 threads will have distinct operations even though they have same source code. Operations will differ based on the input supplied to the source code. As Gilfoy and Svetlin Zarev suggested, I am using one implementation that is perfectly working now. Thanks for your comments though. — Karthikeyan G
– Karthikeyan G, Commented Jun 1, 2016 at 16:39

Svetlin Zarev · Accepted Answer · 2016-05-31 08:20:02Z

2

I think you have your threading model wrong. Currently you want to share which is always wrong in a parallel environment. What you should do instead is to create a pipeline. To do so you have several options.

Use a (blocking/concurrent) queue to pass the objects between threads. In this scenario you have a fixed number of worker threads of each stage of the pipeline that are always running and share the queue. On one side you have the producer threads, which when they finish put() the object to the queue and then one of the consumer threads take() the object and starts to process it.
Use a threadpool to act as the queue from (1). Instead of doing put() on a shared queue you do submit()/execute() to a shared or a dedicated thread pool to pass the object to the next stage of the pipeline. The downside of this approach is that the current stage have to know the exact pool to which to submit the task and also to know how to create the runnable which is to be submitted.

PS: In my answer I assume that Thread-6 performs different kind of operations than the previous 5 threads. If this assumption is incorrect - solution (1) still stands correct and is the way forward.

edited May 31, 2016 at 8:20

answered May 31, 2016 at 7:38

Svetlin Zarev

15.8k4 gold badges60 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Karthikeyan G Over a year ago

Thanks. It works. You are correct. Each thread will perform different kind of operation.

Community · Accepted Answer · 2017-04-12 07:31:17Z

1

Seems to me you are talking about Object pool pattern. Object pools are containers which contain a specified amount of objects. When an object is taken from the pool, it is not available in the pool until it is put back.

Pools should be used in cases such as:

High-frequency using of the same objects
Objects are very big and consume much memory
Objects need much time for initialization
Objects use massive IO operations (Streams, Sockets, DB, etc.)
Objects are not thread-safe

Some publications do not recommend using object pooling, especially for objects that only use memory and hold no external resources. Related criticism question.

Depends why do you need this pattern. Here is implementation in Java.

edited Apr 12, 2017 at 7:31

CommunityBot

11 silver badge

answered May 31, 2016 at 7:49

Abylay Sabirgaliyev

7369 silver badges18 bronze badges

Comments

dev · Accepted Answer · 2016-05-31 08:41:26Z

1

You can define the number of threads you want to run in parallel. For example,

ExecutorService executor = Executors.newFixedThreadPool(20);

Then you can write a for loop to processing the list of objects.

for(Object obj: objList) {
    Runnable thread = new MyThreadImpl(obj);
    executor.execute(thread);
}

This should work perfectly fine as per your

edited May 31, 2016 at 8:41

answered May 31, 2016 at 7:37

dev

1,3553 gold badges20 silver badges40 bronze badges

Comments

kan · Accepted Answer · 2016-05-31 07:37:40Z

0

I doubt there is a standard implementation in JDK, but you could build your own object pool depending on your requirements or probably look into a library, e.g. https://commons.apache.org/proper/commons-pool/

answered May 31, 2016 at 7:37

kan

29.2k7 gold badges75 silver badges112 bronze badges

Comments

Anano · Accepted Answer · 2016-05-31 07:38:08Z

0

Using Reentrant Locks

private static synchronized ReentrantLock getLock(Long id) {
    if (!locks.containsKey(id)) {
        locks.put(id, new ReentrantLock());
    }
    return locks.get(id);
}

private void lock(Long id) {
    ReentrantLock lock = getLock(id);
    lock.lock();
}

private void unlock(Long id) {
    ReentrantLock lock = getLock(id);
    if (lock.isHeldByCurrentThread()) {
        lock.unlock();
    }
}

answered May 31, 2016 at 7:38

Anano

211 silver badge7 bronze badges

Collectives™ on Stack Overflow

How to share a list of objects for parallel processing in Java?

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related