Multithreaded debugging in java

Question

I have a program that runs about 50 threads. I employ a producer consumer design pattern to communicate data between the threads. After the program has been running for a while, sometimes it freezes due to one of the BlockingQueue's I use to distribute data between the threads becomes full, and therefore the main distribution part of the program blocks when it tries to add data to this BlockingQueue. In other words, one of the threads stops for some reason and then the blockingQueue it uses to receive data becomes full.

How do I go about debugging this in an efficient manner? I have tried surrounding the content in all run() methods with catch(Exception e), but nothing is ever thrown. I develop in Java/IntelliJ.

Any thoughts, ideas or general guidelines?

Simplify your program down to remove bits which don't cause the problem. Your problem appears to be the consumer of the queue which is filling up. Why isn't that consumer reading fast enough? Hvae you considered using an ExecutorService which wraps a queue and a thread pools. Often this is easier to use than writing your own thread pool and queue. — Peter Lawrey
– Peter Lawrey, Commented Apr 16, 2015 at 13:36
You could use some tool like jstack (or jconsole) to produce a thread dump and see what your consumer threads are doing at that time. Maybe you have a deadlock which is utterly hard to reproduce on a step-by-step basis — Claudio
– Claudio, Commented Apr 16, 2015 at 13:37
The program is pretty big which makes it hard to isolate relevant code. @Claudio The weird thing is that when I do a thread dump, all I see is threads being blocked by a queue. — user3607022
– user3607022, Commented Apr 16, 2015 at 14:16
I have tried surrounding the content in all run()-methods with catch(Exception e), but nothing is ever thrown. Just a thought: OutOfMemoryError is not an Exception, and so it will not be caught by your handler. How do you know that it wasn't an OOM or some other non-Exception Throwable? — Solomon Slow
– Solomon Slow, Commented Apr 16, 2015 at 14:39

Freiheit · Accepted Answer · 2015-04-16 19:32:48Z

2

"Debug it" by using a logger. I like SLF4J.

Set up log.debug statements before and after each critical operation. Use log.entering and log.exiting calls at the start and end of each method.

While you are 'debugging' you'll run your application with the logger set to a very low level (FINEST) then run your application and watch the logging statements to learn when it fails and what the state is when it fails.

Since you're worried about a threading issue, make sure your log format includes the thread name or number.

edited Apr 16, 2015 at 19:32

answered Apr 16, 2015 at 16:13

Freiheit

8,8156 gold badges64 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

alain · Accepted Answer · 2015-04-16 19:41:54Z

1

general guidelines?

I don't know if this applies to your situation, but a very important guideline is to never have locks being taken in different orders.

An example:

Thread 1:

ResourceA.lock();
ResourceB.lock();
...
ResourceB.unlock();
ResourceA.unlock();

Thread 2:

ResourceB.lock();
ResourceA.lock();
...
ResourceA.unlock();
ResourceB.unlock();

Now if thread 1 is interrupted when it already owns ResourceA but not yet ResourceB, and thread 2 is allowed to run, thread 2 will take ResourceB. Then thread 1 owns ResourceA and waits for ResourceB, and thread 2 owns ResourceB and waits for ResourceA, so you have a deadlock.

edited Apr 16, 2015 at 19:41

answered Apr 16, 2015 at 19:24

alain

12.1k2 gold badges34 silver badges51 bronze badges

Collectives™ on Stack Overflow

Multithreaded debugging in java

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related