avoiding multiple locks while protecting multiple resources

Question

Suppose I need to read/write 1000 files from multiple threads (i.e. one file could be written from multiple threads). I don't want to have one lock to protect all files, since that would be slow. All I can think of is to have a List<object> to hold 1000 locks and do something like lock(list[i]) { write to i'th file}. Is that the right way to do that?

This would be the approach:

    static object _list_lock = new object();
    static List<object> locks = new List<object>();

    public static void Main(string[] args)
    {
        for(int i=0; i<1000; i++)
            locks.Add(new object());

        var tasks = new List<Task>();
        for(int i=0; i<15000; i++)
        {
            var t = Task.Run(() =>
                             {
                                 int file = (new Random()).Next(0, 1000);
                                 object l;
                                 lock(_list_lock)
                                 {
                                     l = locks[file];
                                 }
                                 lock(l)
                                 {                                         
                                     //write to file 'file' 
                                 }
                             });
            tasks.Add(t);
        }
        tasks.ForEach(f => f.Wait());
    }

You might not want 1000's, as you might reach other limits (file handles, physical IO, memory, etc) but you could try running several of these in parallel. Of course, you also need a lock for managing your list of locks :-) The .Net Framework already includes a construct that might help with the type of activity you are looking at - the ReaderWriterLock; msdn.microsoft.com/en-us/library/… — dash
– dash, Commented Apr 3, 2014 at 14:31

WhoIsRich · Accepted Answer · 2014-04-03 14:50:04Z

2

If you have a List<> of file paths, which is checked before reading or writing, I would think it only need to be as big as the number of threads running.

You just need to make sure that the class that adds and removes entries from the List is multi-thread safe so that two threads cannot add the same path at the same time.

answered Apr 3, 2014 at 14:50

WhoIsRich

4,1731 gold badge35 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

avoiding multiple locks while protecting multiple resources

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related