whether a shell script can be executed if another instance of the same script is already running

Question

I have a shell script which usually runs nearly 10 mins for a single run,but i need to know if another request for running the script comes while a instance of the script is running already, whether new request need to wait for existing instance to compplete or a new instance will be started.

I need a new instance must be started whenever a request is available for the same script.

How to do it...

The shell script is a polling script which looks for a file in a directory and execute the file.The execution of the file takes nearly 10 min or more.But during execution if a new file arrives, it also has to be executed simultaneously.

the shell script is below, and how to modify it to execute multiple requests..

#!/bin/bash

while [ 1 ]; do
 newfiles=`find /afs/rch/usr8/fsptools/WWW/cgi-bin/upload/ -newer /afs/rch/usr$
 touch /afs/rch/usr8/fsptools/WWW/cgi-bin/upload/.my_marker

 if [ -n "$newfiles" ]; then
   echo "found files $newfiles"
   name2=`ls /afs/rch/usr8/fsptools/WWW/cgi-bin/upload/ -Art |tail -n 2 |head $
    echo " $name2 "
     mkdir -p  -m 0755 /afs/rch/usr8/fsptools/WWW/dumpspace/$name2
     name1="/afs/rch/usr8/fsptools/WWW/dumpspace/fipsdumputils/fipsdumputil -e -$
  $name1
  touch /afs/rch/usr8/fsptools/WWW/dumpspace/tempfiles/$name2
   fi
   sleep 5
   done

Thanks for adding the script. Obviously it doesn't work yet, but I also can't figure out what it's actually for. Things seems to get really wonky 6 lines from the bottom, where you assign name1 without ever closing the quote. Was that a copy-and-paste error? Can you review and fix it, and explain the context for this script and what it's supposed to achieve? — ghoti
– ghoti, Commented Feb 27, 2016 at 23:42
Actually the script looks for new file in a directory and after that it has to create a new directory with the name of new file.The name1 is nothing but a calling a shell script named as fipsdumputil with aruguments, it nearly takes 10min to complete execution and during that time if any new file comes it also has to be executed ,how it can be done. — DDK
– DDK, Commented Feb 28, 2016 at 3:49
The fipsdumputil(name1) nearly takes 10 min to complete its execution, whether it can be separated from the current script to a new script for it alone ,so that mulitple instance for new file can be made run. — DDK
– DDK, Commented Feb 28, 2016 at 3:54

ghoti · Accepted Answer · 2016-03-03 12:53:12Z

2

When writing scripts like the one you describe, I take one of two approaches.

First, you can use a pid file to indicate that a second copy should not run. For example:

#!/bin/sh

pidfile=/var/run/$(0##*/).pid

# remove pid if we exit normally or are terminated
trap "rm -f $pidfile" 0 1 3 15

# Write the pid as a symlink
if ! ln -s "pid=$$" "$pidfile"; then
  echo "Already running. Exiting." >&2
  exit 0
fi

# Do your stuff

I like using symlinks to store pid because writing a symlink is an atomic operation; two processes can't conflict with each other. You don't even need to check for the existence of the pid symlink, because a failure of ln clearly indicates that a pid cannot be set. That's either a permission or path problem, or it's due to the symlink already being there.

Second option is to make it possible .. nay, preferable .. not to block additional instances, and instead configure whatever it is that this script does to permit multiple servers to run at the same time on different queue entries. "Single-queue-single-server" is never as good as "single-queue-multi-server". Since you haven't included code in your question, I have no way to know whether this approach would be useful for you, but here's some explanatory meta bash:

#!/usr/bin/env bash

workdir=/var/tmp   # Set a better $workdir than this.

a=( $(get_list_of_queue_ids) )  # A command? A function? Up to you.

for qid in "${a[@]}"; do

  # Set a "lock" for this item .. or don't, and move on.
  if ! ln -s "pid=$$" $workdir/$qid.working; then
    continue
  fi

  # Do your stuff with just this $qid.
  ...

  # And finally, clean up after ourselves
  remove_qid_from_queue $qid
  rm $workdir/$qid.working

done

The effect of this is to transfer the idea of "one at a time" from the handler to the data. If you have a multi-CPU system, you probably have enough capacity to handle multiple queue entries at the same time.

edited Mar 3, 2016 at 12:53

answered Feb 25, 2016 at 13:05

ghoti

47.2k8 gold badges71 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

DDK Over a year ago

Actually I need to run multiple instance of same script with different data on requests. is it possible

ghoti Over a year ago

Anything is possible. If you want help with your script, ask a question that includes the script. My second code sample above demonstrates how you might write something that can be run multiple times while clearing entries from the same queue.

DDK Over a year ago

I have added the script to the question, can you tell me how to modify it so that i can run multiple instances

DDK Over a year ago

Can you tell how to modify so that it can accept multiple requests at the same time

ghoti Over a year ago

@DDK, I don't know enough about your environment to be able to recommend the "best" solution. But here's an approach that I used to solve what might have been a similar problem.

Community · Accepted Answer · 2017-05-23 12:31:05Z

1

ghoti's answer shows some helpful techniques, if modifying the script is an option.

Generally speaking, for an existing script:

Unless you know with certainty that:

the script has no side effects other than to output to the terminal or to write to files with shell-instance specific names (such as incorporating $$, the current shell's PID, into filenames) or some other instance-specific location,
OR that the script was explicitly designed for parallel execution,

I would assume that you cannot safely run multiple copies of the script simultaneously.

It is not reasonable to expect the average shell script to be designed for concurrent use.

edited May 23, 2017 at 12:31

CommunityBot

11 silver badge

answered Feb 25, 2016 at 13:17

mklement0

453k68 gold badges729 silver badges989 bronze badges

Comments

user1934428 · Accepted Answer · 2016-02-25 12:44:52Z

0

From the viewpoint of the operating system, several processes may of course execute the same program in parallel. No need to worry about this.

However, it is conceivable, that a (careless) programmer wrote the program in such a way that it produces incorrect results, when two copies are executed in parallel.

answered Feb 25, 2016 at 12:44

user1934428

22.8k9 gold badges57 silver badges109 bronze badges

2 Comments

Cindy Meister Over a year ago

This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review

user1934428 Over a year ago

@CindyMeister: I think it does answer the question. If I understood the question right, the OP wanted to know, whether a script can be run while another instance of the same script is already running. The answer is: From the viewpoint of the OS, it is possible. This is what I answered. Did I miss something?

Collectives™ on Stack Overflow

whether a shell script can be executed if another instance of the same script is already running

3 Answers 3

5 Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related