Multple writer single reader synchronization in multiple process environment?

Question

I have a project in which several writers(using multi thread) write using one process and single reader(Single thread) reads this write through another process. Writer and reader keep doing the same. Writer writes variable lines of data in every write so i cannot use buffer. I think to use either memory map files or single file in which i can write this.

I am confused for synchronization. As am doing this in 2 process so cannot use condition variables so need to use semaphore only.

Now if i use semaphore then several writer write at same time so its overwrite previous data write by another writer and when reader reads it will read only latest write and i will lost all previous write.I want reader should read every write happened. Some delay is acceptable.

I can also try one thing write all in a file then sleep threads for some time and read them during that sleep but by doing this i may lost some data which may arrived during sleep time.

Any idea how to implement the same? Any help is great. I am using C and Linux platform. So kindly suggest related to C and Linux/Unix.

Update1: I have done following steps for semaphore:

                       semget(key,2,flag)  //define 2 semaphore
         semctl(semid,0,SETVAL,0)  //Initialize value for semaphore 0
         semctl(semid,1,SETVAL,0)  //Initialize value for semaphore 0

         writer_lock(semid)  //writer lock
              struct sembuf action[2] ;
            action[0].sem_num = 0;
            action[0].sem_flg = SEM_UNDO;
            action[0].sem_op  = 0 ;
            action[1].sem_num = 1;
            action[1].sem_flg = SEM_UNDO;
            action[1].sem_op  = +1 ;
              semop(sem_id,action,2);

          release_writer_lock
              struct sembuf action[1] ;
            action[0].sem_num = 1;
            action[0].sem_flg = SEM_UNDO;
            action[0].sem_op  = -1 ;
             semop(sem_id,action,1);

         reader_lock(semid)
               struct sembuf action[2] ;
            action[0].sem_num = 1;
            action[0].sem_flg = SEM_UNDO;
            action[0].sem_op  = 0 ;
            action[1].sem_num = 0;
            action[1].sem_flg = SEM_UNDO;
            action[1].sem_op  = +1 ;
             semop(sem_id,action,2) 


          release_reader_lock
              struct sembuf action[1] ;
            action[0].sem_num = 0;
            action[0].sem_flg = SEM_UNDO;
            action[0].sem_op  = -1 ;
             semop(sem_id,action,1);

        ** writer end***
        writer_lock(semid)
          /* my work*/
        release_writer_lock(semid)


        ** reader end***
        reader_lock(semid)
          /* read and clear */
        release_reader_lock(semid)

caf · Accepted Answer · 2013-11-12 06:13:15Z

1

You'll need to use a pair of semaphores - one to block the reader until a writer has created something to read, and one to block the writers until the reader has read the existing messages.

Initialise the read semaphore to 0 and the write sempahore to 1. The reader does:

sem_wait(&read_sem);
/* Read and remove message */
sem_post(&write_sem);

The writers do:

sem_wait(&write_sem);
/* Write message */
sem_post(&read_sem);

answered Nov 12, 2013 at 6:13

caf

241k42 gold badges344 silver badges479 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Tushar Goel Over a year ago

I have made some changes in my post, can you suggest should i use it instead of using sem_wait and sem_post. Also please suggest if i use normal file and open several session for writing will it make some problem? or should i use momory mapping files?

caf Over a year ago

Your code uses the old-style SysV semaphores, I recommend using POSIX semaphores instead (sem_init(), sem_wait(), sem_post()).

caf Over a year ago

You also have the -1 and +1 values the wrong way around - the "lock" operations should subtract one, and the "unlock" operations should add one. release_writer_lock and release_reader_lock operate on the wrong semaphores.

Tushar Goel Over a year ago

Gotcha.. Thanks for the pointing error. I will correct it and also try to implement with POSIX std though i dont have idea how to use it. Will check some tutorial.

caf Over a year ago

@TusharGoel: You can use the SysV sempahores if you want - note that a sem_op of +1 corresponds to a sem_post(), and a sem_op of -1 corresponds to a sem_wait().

|

ravi · Accepted Answer · 2013-11-12 15:02:51Z

0

Note:
1. If the reader process is reading, then no writer process should write.
2. If any writer thread is writing, then reader process can not read as the source file(as you mentioned) is busy.

So, there should be just one synchronization variable(say sem_id) which should be used by both- reader process as well as writer process. Also, all the writer threads should use this variable to avoid overwriting.

writer process:

 writer_thread1 
  {
   writer_lock(sem_id);
     write the data
   release_writer_lock(sem_id);
  }
 writer_thread2 
  {
   writer_lock(sem_id);
     write the data
   release_writer_lock(sem_id);
  }

reader process

  reader 
  {
   reader_lock(sem_id);
     read the data
   release_reader_lock(sem_id);
  }

EDIT:
I am sorry to ignore the message slot limit. But I suggested this solution thinking that you are using a shared file(in which you are writing to and reading from) as you mentioned that in your question. Also, your question mentioned that it has several writer threads, though it is very clear that its not possible to write to a shared location simultaneously.
If there is just one message slot, then it can only be done as mentioned by @caf. But then, I think there is no use of using multiple writer threads because any writer thread can only write data when reader process has read it.

edited Nov 12, 2013 at 15:02

answered Nov 12, 2013 at 6:39

ravi

9282 gold badges6 silver badges15 bronze badges

4 Comments

caf Over a year ago

The problem with this solution is that the subsequent writers will overwrite earlier unread messages - the OP implies that there is only one message slot available, and that subsequent writers should wait until the prior written message has been read.

Tushar Goel Over a year ago

@caf i was implementing the same method. As per you it will not work properly. Need to use POSIX way but is there any way we can correct this ?

ravi Over a year ago

@caf Sorry, I ignored that. Edited now. As your solution uses two semaphores, it seemed to me that a writer thread can only write when the reader process has read the previous one. So, despite of having several writers, we cant use them as only reader process has sem_post(&read_sem) on which each writer thread is waiting.

caf Over a year ago

@RaviKant: Yes, but it doesn't necessarily follow that there is no point in having multiple writers. Perhaps the writers are doing some long-running, resource intensive task and passing the results to a single reader, where the reader can process the results much quicker than a writer can generate the results.

Collectives™ on Stack Overflow

Multple writer single reader synchronization in multiple process environment?

2 Answers 2

7 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related