Parallelism - Shell scripting

Question

I would like to run two commands in my script. The fist runs a test and the second queries this test to obtain a test id.

Now, the problem is that the second command works ONLY WHILE the first command is running (and of course, after it started). So i can't run it sequentially. As far as i can understand, i need some sort of parallelism, which should allow the 2 processes to run in parallel. This would allow the second command to obtain the ID from the first, as it is still running.

Point is: i don't have and i can't install Parallel on this machine. I saw also xargs, but doesn't seem to suite for this task. Any ideas?

The command looks something like this:

run test | list test running

Thanks

Edit This is how the report looks like. It has many different parameters but the first one is the one i need.

ID             | 94503
Name           | Test 
...

To grab the ID i use:

sed -n '1p' | sed 's/^.\{,17\}//' > test_id.dat

Which give me back the ID number.

Piping won't work. The point is that after run test, the problem in fact is that when I start the test, the "shell remains busy" with the test execution. The second command, on the other hand, can retrieve this ID only while the test is running. Example: if I start the test from two separate sessions (one local, one ssh) I can get the expected result.

Edit 2 Some additional details. In order to obtain the "test id", i've to run a commnad which lists all current running tests (list test running). I follow the advices and wrote something like this:

#!/bin/bash
list test running > test_running.dat | {
    run test
    cat test_running.dat | sed -n '1p' | sed 's/^.\{,17\}//' > test_id.dat
    cat test_id.dat
}

The problem here (no matter which command i put first) is that i get as result in the file "No test running". Meaning that the list test running gets executed to early or to late, but in any case not WHILE run test is running.

Can you provide a sample output of run test? Where's the ID in it? — Fravadona
– Fravadona, Commented Apr 11, 2022 at 15:19
You really need to provide more details. How does the 2nd obtain the id? What is the IPC? (Does it attach to a socket, or some shared memory, or does it read from a fifo?). It seems you can probably do exactly what you wrote: command1 | command2 to get the parallelism you want. Why does that pipeline not work? — William Pursell
– William Pursell, Commented Apr 11, 2022 at 15:53
What are the commands that you want to run once you got the ID? — Fravadona
– Fravadona, Commented Apr 11, 2022 at 19:22
@Fravadona the idea is 1) start the test 2) AFTER the test started but WHILE is running, i want to get the ID 3) Parse the ID (since i won't get only the ID but a whole output) 4) save the result in a txt or dat file — t30_9
– t30_9, Commented Apr 19, 2022 at 8:54

Fravadona · Accepted Answer · 2022-04-19 09:32:14Z

1

update

From what I understand the output of run test is irrelevant because you'll get the ID with the list test running command.

I would say that you can simply wait that list test running gives you the ID. Here I use a bash regex based on "how the report looks like" so you might have to adjust it.

#!/bin/bash

run test &

# bash regex for capturing the test ID from the output of 'list test running'
# REMARK: it is based on "how the report looks like" in your question
regex='^ID *\| *([0-9]+)'

while ! [[ $(list test running) =~ $regex ]] && jobs -rp | awk 'END{exit(NR==0)}'
do
    sleep 1
done

test_id=${BASH_REMATCH[1]}

[[ $test_id ]] && echo "I got the test-ID: $test_id"

wait

old answer

So run test starts by outputting a line with the ID and then continue computing and outputting whatever it might be?
It's not clear how you'll use that ID but the following code should show you a way to get what you want and run a command with it while run test continue executing:

#!/bin/bash

run test | {
    IFS= read -r line          # catch the first line (containing the ID)

    obtain test "${line:17}" & # launch the other command as a sub-process

    printf '%s\n' "$line"      # print the first line that was read
    cat                        # continue reading & printing the output
}

^{remark: You also need to disable any output buffering that run test might have.}

edited Apr 19, 2022 at 9:32

answered Apr 11, 2022 at 15:40

Fravadona

17.6k1 gold badge29 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

t30_9 Over a year ago

thank you very much. Still some issues with that. Please have a look to the last edit.

Fravadona Over a year ago

I see. As you don't even need to parse the output of run test to get your ID then your use-case is completely different; just start your script with run test &, that's all you have to do

t30_9 Over a year ago

@Fravedona sounds good BUT i still get "no tests running" since it gets executed too soon (when the test didn't even started yet). Could something like run test & sleep 1 & list test running work?

t30_9 Over a year ago

i test it out. IT WORKS! But: i can't kill the process (it's not required for now, but i can't immagine that i couldn't stop the program) and i updated it to save the ID in a file. For the rest, perfect, thank you very much!

t30_9 Over a year ago

just one small question: is it preferable to use sed (in an efficient way, not like in my code) or as you did with regex?

Collectives™ on Stack Overflow

Parallelism - Shell scripting

1 Answer 1

update

old answer

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

update

old answer

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related