Logging in a multithreading application

Question

I have created a multi-threaded java application and the classes are like this,

public class Logic {

public void method1() {

log.info("Inside method 1");

}

public void method2() {

log.info("Inside method 2");

}

...
.... //and so on

}

The input to the class will be a file and each file will be picked up by a single thread.

Now, say I have 5 input files and from the main() when I invoke the Logic class, I have 5 threads running concurrently in the Logic class and updating the same Log file.

I want to log the information of each input file separately.

Like this,

12-04-2013 com.example.Logic: Inside method 1
12-04-2013 com.example.Logic: Inside method 2

12-04-2013 com.example.Logic: Inside method 1
12-04-2013 com.example.Logic: Inside method 2

12-04-2013 com.example.Logic: Inside method 1
12-04-2013 com.example.Logic: Inside method 2

Whereas now it is liek this,

12-04-2013 com.example.Logic: Inside method 1
12-04-2013 com.example.Logic: Inside method 1
12-04-2013 com.example.Logic: Inside method 1

12-04-2013 com.example.Logic: Inside method 2
12-04-2013 com.example.Logic: Inside method 2
12-04-2013 com.example.Logic: Inside method 2

12-04-2013 com.example.Logic: Inside method 3
12-04-2013 com.example.Logic: Inside method 3
12-04-2013 com.example.Logic: Inside method 3

To summarize, I need the logging to be sequential but the processing to be concurrent.

I dont want to try a StringBuffer to save all the values and log it finally or use a seperate log file or reduce the thread count to one.

Thanks in advance!!!!

Stephen C · Accepted Answer · 2013-12-04 11:48:20Z

Basically, what you are saying is that you want the log file entries to be ordered by some kind of request identifier rather than by the (normal) order in which they are created.

I think @RossDrew's approach is the roughly the right one.

What you would need to do is to write a custom appender (e.g. by extending AppenderSkeleton) that accumulates the events in a buffer until it has received all events in a "set" (e.g. for a specific request). Then it should write all events in the "set" to each of its child appenders.

You would need to deal with cases where event sets are never completed; e.g. because the last method was never called due to an earlier exception. If you are not careful, that would lead to a storage leak in your log event buffering code.

You might be able to implement this using AsyncAppender as a base-class, but there might be issues. (You'd need to understand how that class is implemented ... beyond what the javadoc says.)

Ross Drew · Accepted Answer · 2013-12-04 11:30:16Z

0

Use Log4j, it has an AsyncAppender which takes care of this complicated issue

answered Dec 4, 2013 at 11:30

Ross Drew

8,2563 gold badges44 silver badges53 bronze badges

Comments

Mex · Accepted Answer · 2013-12-04 11:35:49Z

0

For a long running program this will be impractical. I would suggest having a log file for each thread/file, and concatenating the files together when reading them.

answered Dec 4, 2013 at 11:35

Mex

5084 silver badges15 bronze badges

Comments

TwoThe · Accepted Answer · 2013-12-04 12:10:12Z

What you ask is logically impossible, as the only moment where the logger can accurately sort the log events is once the program terminates (and therefore logging is complete). At any point before, it can happen that a sub-sequent log call will mess up the sequential order. You need to either live with the output as you have it, or you need to define sync points, like once a second or once a minute.

For sync points, if you use the standard logging facility, you would have to write a handler that first collects all output in a PriorityQueue with a given sorting function that includes the time and thread source (e.g. collected throuth Thread.currentThread()) and then periodically outputs the oldest messages in that queue, now sorted by thread and time and therefore semi-sequential.

Collectives™ on Stack Overflow

Logging in a multithreading application

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related