1

Is there a way to stop a ListIterator from throwing a ConcurrentModificationException? This is what I want to do:

  • Create a LinkedList with a bunch of objects that have a certain method that is to be executed frequently.
  • Have a set number of threads (say N) all of which are responsible for executing the said method of the objects in the LinkedList. For example, if there are k objects in the list, thread n would execute the method of the n-th object in the list, then move on to n+N-th object, then to n+2N-th, etc., until it loops back to the beginning.

The problem here lies in the retrieval of these objects. I would obviously be using a ListIterator to do this work. However, I predict this will not get very far, thanks to the ConcurrentModificationException that will be thrown according to the documentation. I want the list to be modifiable, and for the iterators to not care. In fact, it is expected that these objects will create and destroy other objects in the list. I've thought of a few work-arounds:

  • Create and destroy a new iterator to retrieve the object at the given index. However, this is O(n), undesirable.
  • Use an ArrayedList instead; however, this is also undesirable, since deletions are O(n) and there are problems with the list needing to expand (and perhaps contract?) from time to time.
  • Write my own LinkedList class. Don't want to.

Thus, my question. Is there a way to stop a ListIterator from throwing a ConcurrentModificationException?

7
  • 1
    Why not consider using a BlockingQueue - docs.oracle.com/javase/6/docs/api/java/util/concurrent/… Commented Jul 19, 2012 at 3:41
  • Why do you have to use iterators? Commented Jul 19, 2012 at 3:47
  • I'm not sure how a BlockingQueue would solve my problem... It's crucial to be able to retrieve an arbitrary index in the list, so a queue would not do... Commented Jul 19, 2012 at 4:10
  • I "have" to use iterators because that's the only way I know of that can be used to traverse a native Java LinkedList. If there are other ways to do this, please share. Commented Jul 19, 2012 at 4:12
  • @user1536654 How often do you modify the list (add, set, remove...)? Commented Jul 19, 2012 at 4:30

4 Answers 4

2

You seem concerned with performance. Have you actually measured the performance hit of using an O(n) vs O(1) algorithm? Depending on what you are doing and how frequently you are doing it, it might be acceptable to simply use a CopyOnWriteArrayList which is thread safe. Its iterators are also thread safe.

The main performance drag is on mutative operations (set, add, remove...): a new list is recreated each time.

However, the performance will be good enough for most applications. I would personally try using that, profile my application to check that the performance is good enough, and move on if it is. If it is not, you will need to find other ways.

Sign up to request clarification or add additional context in comments.

3 Comments

I'm not yet at the stage of the development to be implementing this, so I haven't made any performance tests. However, I'd imagine the difference to be noticeable with thousands of objects.
I asked the question in order to get help making a pre-emptive design choice. But I guess it might be best to just experiment for myself. Thanks for your other suggestion too. :)
If it s a few 1000 objects that you traverse every second you should not notice it. If it s 100,000s of objects every 10ms then you will probably need to find something else. But a producer/consumer pattern would seem applicable to your use case.
2

Is there a way to stop a ListIterator from throwing a ConcurrentModificationException?

That you are asking this question this way shows a lack of understanding of how to properly use threads to increase the performance of your application.

The whole purpose of using threads is to divide processing and IO into separate runnable entities that can be executed in parallel -- independent of each other. If you are forking threads to all work on the same LinkedList then you most likely will have a performance loss or minimal gain since the overhead of the synchronization necessary to keep each of the threads' "view" of the LinkedList in sync would counter any gains due to parallel execution.

The question should not be "how to I stop ConcurrentModificationException", it should be "how can I use threads to improve the processing of a list of objects". That's the right question.

To process a collection of objects in parallel with a number of threads, you should be using an ExecutorService thread-pool. You create the pool with something like the following code. Each of the entries in your LinkedList (in this example Job) would then be processed by the threads in the pool in parallel.

// create a thread pool with 10 workers
ExecutorService threadPool = Executors.newFixedThreadPool(10);
// submit each of the objects in the list to the pool
for (Job job : jobLinkedList) {
    threadPool.submit(new MyJobProcessor(job));
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
// wait for the thread-pool jobs to finish
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
synchronized (jobLinkedList) {
    // not sure this is necessary but we need to a memory barrier somewhere
}
...
// you wouldn't need this if Job implemented Runnable
public class MyJobProcessor implements Runnable {
    private Job job;
    public MyJobProcessor(Job job) {
        this.job = job;
    }
    public void run() {
        // process the job
    }
}

1 Comment

Thank you very much for your input. I thought I was going to get a performance boost due to the fact that LinkedList accesses are a very easy task compared to the actual object processes. But maybe not... I guess I'll have to experiment for myself when I get that far into it. I'll keep your suggestion in mind. Thank you very much once again.
0

You could use one Iterator to scan the list, and use an Executor to do the work on each object by passing off to a pool of threads. That's easy. There's overhead in packaging up work units this way. You still have to be careful to use Iterator method to modify the list, only, but maybe that simplifies the problem.

Or can you perform your work in one pass, then list modification in the next?

Can you split into N lists?

4 Comments

I'm not too sure if I'm understanding you correctly, so please forgive me if I'm not. But I think that's basically what I'm doing. I'll have a pool of threads as you called it, and a list of "tasks", which is scanned by an iterator. The problem is, the list of tasks needs to be modifiable: tasks should be allowed to be added and removed by any thread. That is when an iterator will fail and throw an exception, according to the documentation.
As for performing work in one pass, then listing modifications in the next, I think that would have an adverse effect on performance. I don't think I can afford to do that. Splitting into N lists is also something I've thought of, but it would introduce other disadvantages to my system, and in the end I decided that I'd rather write my own LinkedList if it comes to worst instead of splitting the list. Thanks for the suggestions though. I appreciate your time.
In my first comment, I suggest using one Iterator in one thread, but parceling out work to other threads. This is different, and perhaps simpler, than many Iterators in many threads.
Ah, I see what you mean. That is simpler. However, from what I get, I'm afraid it would not work around the exception, because the list will still be externally modified. Is that correct?
0

Please see the answer from @assylias -- his advice is good. I would add that if you decide to write your own linked list class, you need to think very carefully about how to make it thread-safe.

Think about all the ways your list could get mangled if multiple threads tried to modify it simultaneously. Just locking 1 or 2 nodes is not enough -- as an example, take the following list:

A -> B -> C -> D

Imagine that one thread tries to remove B, just as another thread is removing C. To remove B, the link from A needs to "jump" over B to C. But what if C is no longer part of the list by that time? Likewise, to remove C, the link from B needs to be changed to jump to D, but what if B has already been removed from the list by that time? Similar issues arise when nodes are added simultaneously to nearby parts of the list.

If you have 1 lock per node, and you lock 3 nodes when doing a "remove" operation (the node to be removed, and the nodes before and after it), I think it will be thread-safe. You need to also think carefully about which nodes must be locked when adding nodes, and when traversing the list. To avoid deadlocks, you need to make sure to always acquire locks in a constant order, and when traversing the list, you need to use "hand-over-hand" locking (which precludes the use of ordinary Java monitors -- you need explicit lock objects).

2 Comments

LOL. One of the reasons I didn't want to write my own lists when the ones that come with Java are totally functional. I'm 90% sure I would've messed that up the first time around if I were to have to write my own class. Thanks for pointing it out.
Building correct, robust, high-performance multithreaded code can be very, very complicated. Before you even think about trying to build your own highly concurrent list implementation, you should step back and question whether you could achieve the desired results with a different approach.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.