In a Bash for-loop, can I read the previous iteration's stdout as stdin?

Question

I have a bash oneliner where I pipe one image through the same ImageMagick operation several times. I would like to do this as a Bash loop instead, preferably without using temp files.

The oneliner is: cat in.jpg | convert -quality 1 - jpg:- | convert -quality 1 - jpg:- | convert -quality 1 - jpg:- > out.jpg

This one also works: (convert -quality 1 - jpg:- | convert -quality 1 - jpg:- | convert -quality 1 - jpg:-) > out.jpg < in.jpg

In the convert command the - means "read from stdin" and jpg:- means "output in JPEG format to stdout".

I've tried doing cat in.jpg | for x in {1..3}; do convert -quality 1 - jpg:-; done > out.jpg and (for x in {1..3}; do convert -quality 1 - jpg:-; done) < in.jpg > out.jpg, but both give the same error.

I expect to get the output file with all 3 operations applied to it, but instead I get the output file with just one operation applied to it and the following errors, which I take to mean that the first iteration get to read the in.jpg file, but that the following iterations don't get anything on stdin:

convert: no decode delegate for this image format `' @ > error/constitute.c/ReadImage/501.
convert: no images defined `jpg:-' @ error/convert.c/ConvertImageCommand/3210.
convert: no decode delegate for this image format `' @ error/constitute.c/ReadImage/501.
convert: no images defined `jpg:-' @ error/convert.c/ConvertImageCommand/3210.

(I confirmed that it's only the first iteration that touches the output file by running (for x in 90 30 1; do convert -quality $x - jpg:-; done) < in.jpg > out.jpg

I don't feel that I need to explain my art to you, Warren. 😉 — M Kesti Östergren
– M Kesti Östergren, Commented Jan 29, 2019 at 11:37
@MarkSetchell All jokes aside, I'm exploring interesting compression artefacts for artistic and/or comedic purposes.😄 — M Kesti Östergren
– M Kesti Östergren, Commented Jan 29, 2019 at 11:39
@MarkSetchell Experimenting however shows that ImageMagick is to smart to recompress an image when saved with the same quality as it already had. Adding in an intermediate PNG conversion helped. — M Kesti Östergren
– M Kesti Östergren, Commented Jan 29, 2019 at 12:10

chepner · Accepted Answer · 2019-01-29 14:35:36Z

6

You can define a recursive function:

rec_convert () {
  n=$1
  if [ $n -eq 0 ]; then
    cat
  else
    convert -quality 1 - jpg:- | rec_convert $((n - 1))
  fi
}

rec_convert 3 < in.jpg > out.jpg

edited Jan 29, 2019 at 14:35

answered Jan 29, 2019 at 12:33

chepner

538k77 gold badges595 silver badges747 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mark Setchell Over a year ago

Cool - love it!

Poshi Over a year ago

Oh! So elegant!

chepner Over a year ago

Be aware, this probably isn't suitable for large values of n.

Poshi · Accepted Answer · 2019-01-29 10:37:20Z

2

This is an ugly construction (I don't like the eval), but will solve your problem:

cmd=()
for i in {1..3}
do
    cmd+=("convert" "-quality" "1" "-" "jpg:-" "|")
done
unset "cmd[${#cmd[@]}-1]"

eval "< in.jpg ${cmd[@]} > out.jpg"

You create an array calles cmd. Then, you use a loop to add to the array the components of your command and the pipe symbol. After creating this array, you drop the last element (an extra pipe that won't be needed). Finally, you eval the contatenation of the input redirection, the command line created and the output redirection.

The eval is needed because otherwise, the | is considered as a literal and does not create a pipe but is passed as arcument to convert.

answered Jan 29, 2019 at 10:37

Poshi

5,8733 gold badges22 silver badges37 bronze badges

3 Comments

Mark Setchell Over a year ago

Ha ha, same idea!

Paul Hodges Over a year ago

moot point, but if you want to avoid the eval you could pipe the text to | bash. Six of one, half-dozen of the other, as my grandfather used to say.

Poshi Over a year ago

You are right, I can exchange an eval for an extra process started :-)

Mark Setchell · Accepted Answer · 2019-01-29 10:52:23Z

1

Something like this seems to work:

#!/bin/bash

cmd="cat input.jpg "
for ((iter=0;iter<15;iter++)) do
   cmd+=" | convert jpg:- -quality 50 jpg:- "
done
echo $cmd
bash -c "$cmd" > result.jpg

The debug output of the actual command executed is:

cat input.jpg | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:- | convert jpg:- -quality 50 jpg:-

edited Jan 29, 2019 at 10:52

answered Jan 29, 2019 at 10:39

Mark Setchell

210k32 gold badges310 silver badges505 bronze badges

1 Comment

Paul Hodges Over a year ago

An array is better, but bash -c seems like a good plan to me.

jhnc · Accepted Answer · 2019-01-29 11:30:14Z

And another method using eval:

# wrap command to be run into a shell function
# it should read from stdin and write to stdout
pipecmd(){ convert -quality 1 - jpg:-; }

# generate appropriate number of copies
cmds=()
for x in {1..3}; do cmds[$x]=pipecmd; done

# flatten list into a single string
cmdlist="${cmds[@]}"

# insert the pipes
pipeline="${cmdlist// /|}"

# create a shell function that uses this pipeline
# we need eval so $pipeline is expanded properly
eval "runpipeline(){ $pipeline; }"

# or combine the previous two steps
eval "runpipeline2(){ ${cmdlist// /|}; }"

# check what we created
type pipecmd
type runpipeline
type runpipeline2

# do something
runpipeline <in.jpg >out.jpg

# this will hang!
runpipeline in.jpg >out.jpg

# or just use the wrapped command directly
pipecmd <in.jpg | pipecmd | pipecmd >out.jpg

chepner · Accepted Answer · 2019-01-29 12:34:01Z

0

If you really want to capture the intermediate output, you could use <<< and $()

Note that internally bash does use a temporary file for this, but at least you don't have to do the faff, but with the usual caviats if your content contains quotes:

res1=$(cmd1)
res2=$(cmd2 <<< "$res1" )
res3=$(cmd3 <<< "$res2" )

edited Jan 29, 2019 at 12:34

chepner

538k77 gold badges595 silver badges747 bronze badges

answered Jan 29, 2019 at 12:32

Gem Taylor

5,7351 gold badge11 silver badges34 bronze badges

Collectives™ on Stack Overflow

In a Bash for-loop, can I read the previous iteration's stdout as stdin?

5 Answers 5

3 Comments

3 Comments

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

3 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related