1

I use a command which parses video files for certain frames and returning their timecode, when found. At the moment, I have to execute the command, wait, until the values printed to stdout reach the desired position and then abort the execution using Ctrl+C.

As I have to watch the process and to abort the execution in the right moment to get the information I need, I thought, I could automate this to some degree by creating a bash script.

I am not certain, if it can be done in bash, as I don't exactly know, how to abort the execution in connection with the values it writes to stdout.

The output of the command looks like

0.040000
5.040000
10.040000
15.040000
18.060000
(...)

I tried

until [[ "$timecode" -gt 30 ]]; do
  timecode=$(mycommand)
  sleep 0.1
done

echo "Result: $timecode"

or

while [[ "$timecode" -le 30 ]]; do
  timecode=$(mycommand)
  sleep 0.1
done

echo "Result: $timecode"

which both seem to result in the command being executed until it finishes and afterwards the rest of the loop is being processed. But I want to evaluate the output while the command executes and break execution depending on the output.

Additional information

The command has no capability to be stopped at a certain point in the stream. It parses the whole file and gives the results unless signalled to stop. This was my first shot.

The execution time of the command is very long as the files I parse are ~2GB. As I don't need all frames of the file but only a few around a given timecode, I never let it execute until it finished.

The output of the command varies from file to file, so I can't look for an exact value. If I knew the exact value, I probably wouldn't have to look for it.

The destination time code - in the example it is specified by "-gt 30" - is different for every file I will have to parse, so I will have to put this into a command line parameter once the script works. I would also have to make sure to get back more than the last value of the execution but about the last 5 values. For these two I already have Ideas.

I'm totally stuck on that one and have not even an idea what to google for.

Thank you for your input!

Manuel

With the answers of PSkocik and Kyle Burton, I was able to integrate the suggested solution into my script. It doesn't work and I don't see, why.

Here the complete script including the external command providing the output:

 #!/usr/bin/env bash
 set -eu -o pipefail

 parser () {
   local max="$1"
   local max_int

   max_int="${max%.*}"

   while read tc;
     do
       local tc_int
       tc_int="${tc%.*}"
       echo $tc

       if (( "$tc_int" >= "$max_int" )); then
         echo "Over 30: $tc";
         exec 0>&-
         return 0
       fi

     done
 }

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | sed -ne "s/^1|//p" | parser 30

I don't get any output from the "echo $tc" but the ffprobe is running - I can see it in top. It runs until I stop the script using Ctrl+C.

Thank you Kyle for your big efforts in this. I'd never come to such a conclusion. I changed the commandline of ffprobe to your suggestion

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | cut -f2 -d\| | parser 30

and now, I'm getting results while ffprobe runs. But... the way you changed the command returns all frames, ffprobe finds and not only the Keyframes. The original output of the ffprobe command looks like

 1|0.000000
 0|0.040000
 0|0.080000
 0|0.120000
 0|0.160000
 0|0.200000
 (...)

The 0 at the beginning of the line means: this is no keyframe. The 1 at the beginning of the line means: this is a keyframe.

The script is intended to provide only the keyframes around a certain timecode of the video file. The way you changed the command, it now provides all frames of the video file what makes the resulting output useless. It has to be filtered for all lines starting with zero to be dropped.

As I don't exactly understand, why this doesn't work with sed, I can only try to find a solution by try and error, facilitating different tools to filter the output. But if the filtering itself causes the problem, we might have hit a wall here.

Improve this question
18
  • I'm wondering if the read is working. If you change echo $tc to echo tc=$tc do you see lines like tc=? Can you run ffprobe <<with those args>> | sed -ne "s/^1|//p" | head and share that output? I'm wondering now why the read is failing and if ffprobe is printing lines -- so I'm trying to narrow down where the error might be. Commented Jul 26, 2017 at 5:21
  • Changed the echo $tc to echo "Read: $tc" but nothing is printed. # ffprobe Input.avi -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | sed -ne "s/^1|//p" prints 0.040000 5.040000 10.040000 15.040000 18.060000 21.980000 26.980000 Commented Jul 26, 2017 at 8:09
  • Ah! Is the output of ffprobe space instead of newline separated? Let me adjust my example to see if I can get it to translate the spaces into newlines and get closer to what you need. I updated the answer to include the use of tr - give it a try? Commented Jul 26, 2017 at 15:00
  • already thought you might go for this. The return values have been fit into one line by the commenting system. Each value is in a single line. Tried your update and nothing changed. When using your version including the function generating values, I get the values back. Commented Jul 26, 2017 at 19:45
  • Ok, so here's what I think the current facts are about ffprobe: it emits one floating point number per line, we seem to not be able to get those to be read line by line. We're sure it's not going over stderr? How can we narrow things down to just what ffprobe is outputting and how it's outputting it? If you were to take the output you copied above and put it into a text file, then the read loop would work, so what's keeping it from working? Is it possible for you to post a (small) video file somewhere that I could test this script and ffprobe myself? Commented Jul 27, 2017 at 2:53

3 Answers 3

Reset to default
1

If you have process a that's outputting stuff to stdout and process b that reads the outputted stuff via a pipe:

a | b

all b has to usually do to kill a when a certain item is outputted is to close its standard input.

A sample b:

b()
{
    while read w;
        do case $w in some_pattern)exec 0>&-;; esac; 
        echo $w
    done
}

This closing of stdin (filedescriptor 0) will cause the producer process to be killed by SIGPIPE the moment it tries to make its next write.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I currently cannot verify this, as I don't know how to pipe the output of the externally executed program to the internal bash script procedure. The command producing the output is being started from inside the same bash script as the procedure which should evaluate its results and break its execution. And I already have switched to an if statement, as case is evaluating patterns and not numeric values, which makes everything more complicated.
0

I think PSkocik's approach makes sense. I think all you need to do is run your mycommand and pipe it into your while loop. If you put PSkocik's code in a file wait-for-max.sh then you should be able to run it as:

mycommand | bash wait-for-max.sh

After working with M. Uster in comments above, we've come up with the following solution:

#!/usr/bin/env bash
set -eu -o pipefail

# echo "bash cutter.sh rn33.mp4"

# From: https://stackoverflow.com/questions/45304233/execute-command-in-bash-script-until-output-exceeds-certain-value
# test -f stack_overflow_q45304233.tar ||  curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f stack_overflow_q45304233.tar ||  curl -k -O https://84.19.186.119/stack_overflow_q45304233.tar
# test -f rn33.mp4 || curl -k -O https://84.19.186.119/rn33.mp4

function parser () {
  local max="$1"
  local max_int

  # NB: this removes everything after the decimal point
  max_int="${max%.*}"

  # I added a line number so I could match up the ouptut from this function
  # with the output captured by the 'tee' command
  local lnum="0"
  while read -r tc;
    do

      lnum="$(( 1 + lnum ))"

      # if a blank line is read, just ignore it and continue
     if [ -z "$tc" ]; then
       continue
     fi

     local tc_int
     # NB: this removes everything after the decimal point
     tc_int="${tc%.*}"
     echo "Read[$lnum]: $tc"

     if (( "$tc_int" >= "$max_int" )); then
       echo "Over 30: $tc";
       # This closes stdin on this process, which will cause an EOF on the
       # process writing to us across the pipe
       exec 0>&-
       return 0
     fi

    done
}

# echo "bash version:    $BASH_VERSION"
# echo "ffprobe version: $(ffprobe -version | head -n1)"
# echo "sed version:     $(sed --version | head -n1)"

# NB: by adding in the 'tee ffprobe.out' into the pipeline I was able to see
# that it was producing lines like:
#
# 0|28.520000
# 1|28.560000
#
#
# changing the sed to look for any single digit and a pipe fixed the script
# another option is to use cut, see below, which is probalby more robust.

# ffprobe "$1" \
#   -hide_banner \
#   -select_streams v \
#   -show_entries frame=key_frame,best_effort_timestamp_time \
#   -of csv=nk=1:p=0:s="|" \
#   -v quiet 2>&1 | \
#   tee ffprobe.out |
#   sed -ne "s/^[0-9]|//p" | \
#   parser 30


ffprobe "$1" \
    -hide_banner \
    -select_streams v \
    -show_entries frame=key_frame,best_effort_timestamp_time \
    -of csv=nk=1:p=0:s="|" \
    -v quiet 2>&1 | \
    cut -f2 -d\| | \
    parser 30
Improve this answer

3 Comments

Thank you for your example. I managed to integrate this inside my script but it doesn't work yet. I don't get any ouput from the "echo $timecode" so the reading seems not to work as expected. I update the question now.
Note that on the last line you'll need to replace parse-video-frames with whatever mycommand is from your question. Is it possible that it's printing to stderr instead of stdout?
Now I've updated the question including the whole script. You can see, I did it as you proposed and nothing is being redirected to stderr. I call the script by its name and provide a filename as parameter.
0

The answer to my question has finally been found by the help of PSkocik and intense support of Kyle Burton. Thanks to both of you!

I didn't know, that it is possible to pipe the output of commands executed in a script to a function that belongs to the script. This was the first piece of information necessary.

And I didn't know, how to evaluate the piped information inside the function properly and how to signal from inside the function, that the execution of the command generating the values should be terminated.

Additionally, Kyle found, that the filtering I did by piping the original output to sed and the resulting data to the function inside the script prohibited the script to function as designed. I'm still uncertain, why - but it definitively does.

The original command generating the output is now being piped as it is to the internal function of the script. The filtering is being done inside the function to avoid the problem with sed. Now everything works as expected and I can continue completing the script.

This is the working code of the soultion:

 #!/usr/bin/env bash
 set -eu -o pipefail

 function parser () {
   local max="$1"
   local max_int

   max_int="${max%.*}"

   while read tc;
     do

      #If line is empty, continue
      if [ -z "$tc" ]; then
        continue
      fi

      #If first char is 0 (=non-Index Frame), continue
      local iskey="${tc:0:1}";

      if [ $iskey == "0" ]; then
        continue
      fi

      #Return timecode if intended maximum has been reached
      local val="${tc:2:10}"
      local tc_int
      tc_int="${val%.*}"

      if (( "$tc_int" >= "$max_int" )); then
        echo "First index frame at/after given Timecode: $tc";
        exec 0>&-
        return 0
      fi

     done
 }

 ffprobe "$1" -hide_banner -select_streams v -show_entries frame=key_frame,best_effort_timestamp_time -of csv=nk=1:p=0:s="|" -v quiet | parser "$2"

Usage:

 ./script.sh "Name of Movie.avi" 30

where 30 represents the timecode at which the next found index frame is being searched and returned.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.