Expanding a bash function to add input redirection and/or pipes

Question

I'm trying to set up a bash function that annotates the output from commands with some prefix. Currently, I have a bunch of lines of code that look like this:

git pull              2>&1 | sed "s/^/   [git pull] /"
clean_cmake_fortests  2>&1 | sed "s/^/   [cmake] /"
make -j 2             2>&1 | sed "s/^/   [make] /"
docker rmi $(docker images -a -q) 2>&1 | sed "s/^/   [docker rmi] /" | grep "removed" || true

My goal is to replace the 2>&1 | sed "s/^/$ [$1] /" bit with a function so that I can just make the above lines look something like:

git pull              `annotate "git pull"`
clean_cmake_fortests  `annotate "cmake"`
make -j 2             `annotate "make"`
docker rmi $(docker images -a -q) `annotate "docker rmi"` | grep "removed" || true

I defined the function annotate as

function annotate {
   2>&1 | sed "s/^/    [$1] /"
}

But when executing, it has no impact, and the commands all just dump their standard output unmodified. How can I achieve what I'm intending here? I'm going for something akin to C inline macro expansions.

If anyone's curious, the point of this is to let me generate logs like this:

04: Getting proto images
    [docker_get_proto_images] Fetching proto docker images...
    ...
    [docker_get_proto_images] Status: Image is up to date for api:dev-proto
    [docker_get_proto_images] ... Done.
05: Building local docker containers
    [docker_local_build] [sh] Building local docker images...
    [docker_local_build] [sh] NOTE: Odd behaviour may result if using outdated bases...
    [docker_local_build] [sh] Local docker image build complete.
    ...
    [docker_local_build] [sh] For advanced usage, see $ARE_TOP/deployment/docker/README
    [docker_local_build]
06: Running docker-compose
    [docker-compose] Starting docker_datacachedisk_1
    [docker-compose] Starting docker_djangodisk_1
    ...

rather than this:

04: Getting proto images
Fetching proto docker images...
...
Status: Image is up to date for api:dev-proto
... Done.
05: Building local docker containers
[sh] Building local docker images...
[sh] NOTE: Odd behaviour may result if using outdated bases...
[sh] Local docker image build complete.
...
[sh] For advanced usage, see $ARE_TOP/deployment/docker/README

06: Running docker-compose
Starting docker_datacachedisk_1
Starting docker_djangodisk_1
...

Which gets hard to read after a while.

You will rather want something along the lines of annotate $(git pull) "git pull". The way your syntax is written, it would enable git pull to work on the output of annotate rather than the other way, since arguments are evaluated before the main command — Aaron
– Aaron, Commented Feb 11, 2016 at 17:47
@Alain Your approach doesn't work because the output of annotate is substituted after the command line has been parsed. Anything it outputs is passed to the original command as string arguments, not parsed as shell syntax. — chepner
– chepner, Commented Feb 11, 2016 at 17:59

chepner · Accepted Answer · 2016-02-11 18:04:04Z

2

Redirection is tied to a command, so you can't separate 2>&1 from the command it affects. You can define annotate similar to how you are now, though. (Don't use sed, though, because it's exceedingly difficult to include a variable in its command without knowing what delimiter is being used.)

annotate ()
while IFS= read -r line; do
       printf '    [%s] %s\n' "$1" "$line"
done

(Yes, the lack of braces is intentional, though not necessary. The body of a function can be any compound command, not just a brace group.)

Then call it as

git pull &> >( annotate "git pull" )

You could use a simple pipe, git pull |& annotate "git pull", but that runs the command in a subshell, which might not be desirable.

edited Feb 11, 2016 at 18:04

answered Feb 11, 2016 at 17:51

chepner

538k77 gold badges595 silver badges747 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alain Over a year ago

That's the ticket. Thanks!

muru · Accepted Answer · 2016-02-11 18:13:59Z

0

Execute the commands in your function. For example:

annotate () {
    "$@" |& sed "s/^/    [$1] /"
}

Here, write the command as you normally would, just add annotate at the start:

$ annotate ls /
    [ls] bin
    [ls] boot
    [ls] dev
    [ls] etc
    [ls] home
    ...

Awk might be a better bet. If your IFS starts with a space (which, by default, it does), you can get the entire command in using $*:

annotate () {
    "$@" |& awk -v tag="$*" '{printf "\t[%s] | %s\n", tag, $0}'
}

Again:

$ annotate ls /
    [ls /] | bin
    [ls /] | boot
    [ls /] | dev
    [ls /] | etc
    [ls /] | home

Alternatively, you can keep the first argument as the tag, so that you can pass an arbitrary tag:

annotate () {
    "${@:2}" |& awk -v tag="$1" '{printf "\t[%s] | %s\n", tag, $0}'
}

$ annotate "foo bar" ls /
    [foo bar] | bin
    [foo bar] | boot
    [foo bar] | dev
    [foo bar] | etc
    [foo bar] | home
    [foo bar] | lib

edited Feb 11, 2016 at 18:13

answered Feb 11, 2016 at 17:51

muru

4,9811 gold badge38 silver badges84 bronze badges

11 Comments

chepner Over a year ago

annotate "git pull" will attempt to run the command git pull, not git with its first argument pull. annotate git pull will only pass git, not git pull, as the tag to add with sed.

muru Over a year ago

@chepner as I said, write the command as you normally would. That is, without the quotes. That's why the example has ls and / without quotes.

chepner Over a year ago

I added the problem you can encounter if you don't quote it. This is aside from the problem of passing more complicated commands than simply a command name and its arguments to annotate.

muru Over a year ago

@chepner If you try to execute a quoted string as a command, that is taking on more trouble than is worth. unix.stackexchange.com/q/251103/70524

chepner Over a year ago

mywiki.wooledge.org/BashFAQ/050 is relevant here. The point is, there are problems whether or not you quote it.

|

Collectives™ on Stack Overflow

Expanding a bash function to add input redirection and/or pipes

2 Answers 2

1 Comment

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related