2

Hello I am interested in how to pass stdout twice to the same command...

For instance, If I run the following:

seq 5 >a
tac a >b
paste a b

then I get:

1       5
2       4
3       3
4       2
5       1

The following also deliver the same result:

paste <(seq 5) <(seq 5 |tac)

or

seq 5 | paste - <(seq 5 |tac)

I want to use seq 5 only once - perhaps similar to these attempts, which do not work:

seq 5 | paste - <(tac -)

or

seq 5 | tee >(tac 1>&3) | paste - >(3>&1 cat)

I expect some kind of file descriptor manipulation or further process substitution magic will do the trick but I am having trouble getting there.

Improve this question

4 Answers 4

Reset to default
1

You could do something like this:

rootdir=$(mktemp -d)  # Strictly speaking not needed, but we create us a tempdir
mkfifo "$rootdir/pipe"  # create a named pipe
seq 5 | tee "$rootdir/pipe" | paste - <(tac "$rootdir/pipe")
# We could now tee (split) output into that pipe
rm -Rf "$rootdir"  # cleanup

You could also have extra file descriptor involved, but you'd still need something to underpin it with to connect both ends of the stream. Problem is, while you could multiplex the output, on a newly created problem you would still only have one input stream (when piping commands connected to stdout of the preceding process at that). A second transmission path must be added.

tmpfile=$(mktemp); exec 3<>"$tmpfile"; rm "$tmpfile"

Creates a temporary file and opens fd 3 for reading/writing. Once we have a file descriptor, we no longer need name for that file in the file system.

seq 5 | tee /proc/self/fd/3 | paste - <(tac </proc/self/fd/3)

We still use tee to split the output between stdout and fd 3. File descriptors are inherited by child process, so we can refer to them with /proc/self/ (also from perspective of tee and tac).

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Without a trap you will leave the temporary file behind in case of error or an interruption in the first scenario.
@tripleee No I won't, not in the second example unless the shell exits between mktemp and and rm. I am abusing the fact that unlink only removes the reference, but leaves the file as such intact as long as fd is open to it. That could have happened between mktemp and trap as well though, so not really necessary to set it.
In the first scenario specifically, like I said, the rm -Rf won't happen if the script is interrupted. The race condition between creating a directory and setting a trap is usually blithely ignored in practice, although I suppose you could do things the other way around as well.
And in the second, like I said, it's not a concern. So we're in agreement. I'd prefer the dangling file descriptor over trap to be honest, it looks a little neater to me.
Btw. those are not really scripts and are meant as examples on an interactive shell, so the concern (as much as appreciated and not entirely wrong) is even less relevant.
1

There is no way to duplicate a single stream, short of using a tool such as tee. A common arrangement is to use a temporary file.

#!/bin/bash
t=$(mktemp -t pastepaste.XXXXXXXXXX) || exit
trap 'rm -f "$t"' ERR EXIT
tee "$t" |
paste - <(tac "$t")
Improve this answer

Comments

0

The problem you're facing is that each element in a pipeline is executed in a separate subshell, so any newly-redirected FDs by the elements aren't reflected in the parent shell, which is doing all the I/O coordination.

Some other guru might come up with a magic redirect incantation to solve this problem at the parent level, but for clarity, I'd suggest named pipes (a.k.a. FIFOs) instead. Since they exist outside of the shell environment, they're visible to all pipeline elements at all times.

More importantly, with proper naming, "connecting the pipes" becomes a relatively bug-free process, especially as your pipeline "network" gets more complex. No guessing games about which FD carries which output.

For your example, this works seamlessly:

#!/bin/bash
# Make a temp dir...
tmpd=$(mktemp -d)
# ...and clean it up automatically
trap 'rm -fr $tmpd' EXIT

# Create the FIFO...
mkfifo $tmpd/tac.fifo
# ...and PROFIT!
seq 5 | tee >(tac >$tmpd/tac.fifo) | paste - $tmpd/tac.fifo

Further Reading:

Improve this answer

Comments

0

A simple, symmetrical example would be to use two named pipes.

 trap 'rm p1 p2' ERR EXIT
 mkfifo p1 p2

Then you can have tac filter the output of tee into p2

 seq 5 | tee p1 <(tac > p2) > /dev/null &
 paste p1 p2

or filter the contents of p2 before passing to paste.

seq 5 | tee p1 p2 > /dev/null
paste p1 <(tac p2)
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.