2626

Say, I have a script that gets called with this line:

./myscript -vfd ./foo/bar/someFile -o /fizz/someOtherFile

or this one:

./myscript -v -f -d -o /fizz/someOtherFile ./foo/bar/someFile 

What's the accepted way of parsing this such that in each case (or some combination of the two) $v, $f, and $d will all be set to true and $outFile will be equal to /fizz/someOtherFile?

6
  • 2
    For zsh-users there's a great builtin called zparseopts which can do: zparseopts -D -E -M -- d=debug -debug=d And have both -d and --debug in the $debug array echo $+debug[1] will return 0 or 1 if one of those are used. Ref: zsh.org/mla/users/2011/msg00350.html Commented Aug 2, 2016 at 2:13
  • 4
    Really good tutorial: linuxcommand.org/lc3_wss0120.php. I especially like the "Command Line Options" example. Commented Feb 10, 2020 at 18:45
  • I created a script which does it for you, it's called - github.com/unfor19/bargs Commented Aug 7, 2020 at 14:29
  • 2
    See also Giving a bash script the option to accepts flags, like a command? for an elaborate, ad hoc, long and short option parser. It does not attempt to handle option arguments attached to short options, nor long options with = separating option name from option value (in both cases, it simply assumes that the option value is in the next argument). It also doesn't handle short option clustering — the question didn't need it. Commented Oct 9, 2020 at 15:21
  • 1
    This great tutorial by Baeldung shows 4 ways to process command-line arguments in bash, including: 1) positional parameters $1, $2, etc., 2) flags with getopts and ${OPTARG}, 3) looping over all parameters ($@), and 4) looping over all parameters using $#, $1, and the shift operator. Commented Dec 26, 2020 at 21:05

43 Answers 43

1
2
2

I wrote down a script that can assist with parsing command-line arguments easily - https://github.com/unfor19/bargs

Examples

$ bash example.sh -n Willy --gender male -a 99
Name:      Willy
Age:       99
Gender:    male
Location:  chocolate-factory
$ bash example.sh -n Meir --gender male
[ERROR] Required argument: age

Usage: bash example.sh -n Willy --gender male -a 99

--person_name  |  -n  [Willy]              What is your name?
--age          |  -a  [Required]
--gender       |  -g  [Required]
--location     |  -l  [chocolate-factory]  insert your location
$ bash example.sh -h

Usage: bash example.sh -n Willy --gender male -a 99
--person_name  |  -n  [Willy]              What is your name?
--age          |  -a  [Required]
--gender       |  -g  [Required]
--location     |  -l  [chocolate-factory]  insert your location
Sign up to request clarification or add additional context in comments.

Comments

2

I ended up implementing the dash (or /bin/sh) version of the accepted answer, basically, without array usage:

while [[ $# -gt 0 ]]; do
    case "$1" in
    -v|--verbose) verbose=1; shift;;
    -o|--output) if [[ $# -gt 1 && "$2" != -* ]]; then
            file=$2; shift 2
        else
            echo "-o requires file-path" 1>&2; exit 1
        fi ;;
    --)
        while [[ $# -gt 0 ]]; do BACKUP="$BACKUP;$1"; shift; done
        break;;
    *)
        BACKUP="$BACKUP;$1"
        shift
        ;;
    esac
done
# Restore unused arguments.
while [ -n "$BACKUP" ] ; do
    [ ! -z "${BACKUP%%;*}" ] && set -- "$@" "${BACKUP%%;*}"
    [ "$BACKUP" = "${BACKUP/;/}" ] && break
    BACKUP="${BACKUP#*;}"
done

Comments

2

I have browsed through all of the answers to this question. And although some contain wealth of information, I was specifically looking for the answers that allow us to describe the supported command line arguments declaratively and get the help text generated automatically from the spec.

And I found 5 such answers (sorry, if I missed yours):

  1. argbash
  2. getoptions
  3. bashopts
  4. bash_option_parser
  5. bargs

I am also guilty of writing my own command line parser, which seems to be different from all the rest, because I use JSON to describe the supported command line arguments and thus my implementation depends on jq.

Check it out here - https://github.com/MarkKharitonov/bash-parse-command-line-args. The repo contains an example script and the README shows off various invocation scenarios.

Comments

1

Another solution without getopt[s], POSIX, old Unix style

Similar to the solution Bruno Bronosky posted this here is one without the usage of getopt(s).

Main differentiating feature of my solution is that it allows to have options concatenated together just like tar -xzf foo.tar.gz is equal to tar -x -z -f foo.tar.gz. And just like in tar, ps etc. the leading hyphen is optional for a block of short options (but this can be changed easily). Long options are supported as well (but when a block starts with one then two leading hyphens are required).

Code with example options

#!/bin/sh

echo
echo "POSIX-compliant getopt(s)-free old-style-supporting option parser from phk@[se.unix]"
echo

print_usage() {
  echo "Usage:

  $0 {a|b|c} [ARG...]

Options:

  --aaa-0-args
  -a
    Option without arguments.

  --bbb-1-args ARG
  -b ARG
    Option with one argument.

  --ccc-2-args ARG1 ARG2
  -c ARG1 ARG2
    Option with two arguments.

" >&2
}

if [ $# -le 0 ]; then
  print_usage
  exit 1
fi

opt=
while :; do

  if [ $# -le 0 ]; then

    # no parameters remaining -> end option parsing
    break

  elif [ ! "$opt" ]; then

    # we are at the beginning of a fresh block
    # remove optional leading hyphen and strip trailing whitespaces
    opt=$(echo "$1" | sed 's/^-\?\([a-zA-Z0-9\?-]*\)/\1/')

  fi

  # get the first character -> check whether long option
  first_chr=$(echo "$opt" | awk '{print substr($1, 1, 1)}')
  [ "$first_chr" = - ] && long_option=T || long_option=F

  # note to write the options here with a leading hyphen less
  # also do not forget to end short options with a star
  case $opt in

    -)

      # end of options
      shift
      break
      ;;

    a*|-aaa-0-args)

      echo "Option AAA activated!"
      ;;

    b*|-bbb-1-args)

      if [ "$2" ]; then
        echo "Option BBB with argument '$2' activated!"
        shift
      else
        echo "BBB parameters incomplete!" >&2
        print_usage
        exit 1
      fi
      ;;

    c*|-ccc-2-args)

      if [ "$2" ] && [ "$3" ]; then
        echo "Option CCC with arguments '$2' and '$3' activated!"
        shift 2
      else
        echo "CCC parameters incomplete!" >&2
        print_usage
        exit 1
      fi
      ;;

    h*|\?*|-help)

      print_usage
      exit 0
      ;;

    *)

      if [ "$long_option" = T ]; then
        opt=$(echo "$opt" | awk '{print substr($1, 2)}')
      else
        opt=$first_chr
      fi
      printf 'Error: Unknown option: "%s"\n' "$opt" >&2
      print_usage
      exit 1
      ;;

  esac

  if [ "$long_option" = T ]; then

    # if we had a long option then we are going to get a new block next
    shift
    opt=

  else

    # if we had a short option then just move to the next character
    opt=$(echo "$opt" | awk '{print substr($1, 2)}')

    # if block is now empty then shift to the next one
    [ "$opt" ] || shift

  fi

done

echo "Doing something..."

exit 0

For the example usage please see the examples further below.

Position of options with arguments

For what its worth there the options with arguments don't be the last (only long options need to be). So while e.g. in tar (at least in some implementations) the f options needs to be last because the file name follows (tar xzf bar.tar.gz works but tar xfz bar.tar.gz does not) this is not the case here (see the later examples).

Multiple options with arguments

As another bonus the option parameters are consumed in the order of the options by the parameters with required options. Just look at the output of my script here with the command line abc X Y Z (or -abc X Y Z):

Option AAA activated!
Option BBB with argument 'X' activated!
Option CCC with arguments 'Y' and 'Z' activated!

Long options concatenated as well

Also you can also have long options in option block given that they occur last in the block. So the following command lines are all equivalent (including the order in which the options and its arguments are being processed):

  • -cba Z Y X
  • cba Z Y X
  • -cb-aaa-0-args Z Y X
  • -c-bbb-1-args Z Y X -a
  • --ccc-2-args Z Y -ba X
  • c Z Y b X a
  • -c Z Y -b X -a
  • --ccc-2-args Z Y --bbb-1-args X --aaa-0-args

All of these lead to:

Option CCC with arguments 'Z' and 'Y' activated!
Option BBB with argument 'X' activated!
Option AAA activated!
Doing something...

Not in this solution

Optional arguments

Options with optional arguments should be possible with a bit of work, e.g. by looking forward whether there is a block without a hyphen; the user would then need to put a hyphen in front of every block following a block with a parameter having an optional parameter. Maybe this is too complicated to communicate to the user so better just require a leading hyphen altogether in this case.

Things get even more complicated with multiple possible parameters. I would advise against making the options trying to be smart by determining whether the an argument might be for it or not (e.g. with an option just takes a number as an optional argument) because this might break in the future.

I personally favor additional options instead of optional arguments.

Option arguments introduced with an equal sign

Just like with optional arguments I am not a fan of this (BTW, is there a thread for discussing the pros/cons of different parameter styles?) but if you want this you could probably implement it yourself just like done at http://mywiki.wooledge.org/BashFAQ/035#Manual_loop with a --long-with-arg=?* case statement and then stripping the equal sign (this is BTW the site that says that making parameter concatenation is possible with some effort but "left [it] as an exercise for the reader" which made me take them at their word but I started from scratch).

Other notes

POSIX-compliant, works even on ancient Busybox setups I had to deal with (with e.g. cut, head and getopts missing).

Comments

1

The top answer to this question seemed a bit buggy when I tried it -- here's my solution which I've found to be more robust:

boolean_arg=""
arg_with_value=""

while [[ $# -gt 0 ]]
do
key="$1"
case $key in
    -b|--boolean-arg)
    boolean_arg=true
    shift
    ;;
    -a|--arg-with-value)
    arg_with_value="$2"
    shift
    shift
    ;;
    -*)
    echo "Unknown option: $1"
    exit 1
    ;;
    *)
    arg_num=$(( $arg_num + 1 ))
    case $arg_num in
        1)
        first_normal_arg="$1"
        shift
        ;;
        2)
        second_normal_arg="$1"
        shift
        ;;
        *)
        bad_args=TRUE
    esac
    ;;
esac
done

# Handy to have this here when adding arguments to
# see if they're working. Just edit the '0' to be '1'.
if [[ 0 == 1 ]]; then
    echo "first_normal_arg: $first_normal_arg"
    echo "second_normal_arg: $second_normal_arg"
    echo "boolean_arg: $boolean_arg"
    echo "arg_with_value: $arg_with_value"
    exit 0
fi

if [[ $bad_args == TRUE || $arg_num < 2 ]]; then
    echo "Usage: $(basename "$0") <first-normal-arg> <second-normal-arg> [--boolean-arg] [--arg-with-value VALUE]"
    exit 1
fi

Comments

1

Here is my improved solution of Bruno Bronosky's answer using variable arrays.

it lets you mix parameters position and give you a parameter array preserving the order without the options

#!/bin/bash

echo $@

PARAMS=()
SOFT=0
SKIP=()
for i in "$@"
do
case $i in
    -n=*|--skip=*)
    SKIP+=("${i#*=}")
    ;;
    -s|--soft)
    SOFT=1
    ;;
    *)
        # unknown option
        PARAMS+=("$i")
    ;;
esac
done
echo "SKIP            = ${SKIP[@]}"
echo "SOFT            = $SOFT"
    echo "Parameters:"
    echo ${PARAMS[@]}

Will output for example:

$ ./test.sh parameter -s somefile --skip=.c --skip=.obj
parameter -s somefile --skip=.c --skip=.obj
SKIP            = .c .obj
SOFT            = 1
Parameters:
parameter somefile

2 Comments

You use shift on the known arguments and not on the unknown ones so your remaining $@ will be all but the first two arguments (in the order they are passed in), which could lead to some mistakes if you try to use $@ later. You don't need the shift for the = parameters, since you're not handling spaces and you're getting the value with the substring removal #*=
You're right, in fact, since I build a PARAMS variable, I don't need to use shift at all
1

Simple and easy to modify, parameters can be in any order. this can be modified to take parameters in any form (-a, --a, a, etc).

for arg in "$@"
do
   key=$(echo $arg | cut -f1 -d=)`
   value=$(echo $arg | cut -f2 -d=)`
   case "$key" in
        name|-name)      read_name=$value;;
        id|-id)          read_id=$value;;
        *)               echo "I dont know what to do with this"
   ease
done

1 Comment

In this script, ease is incorrect and should be replaced with esac to close the case block
1

I use it to iterate over key => value from the end. A first optional argument is caught after the loop.

Usage is ./script.sh optional-first-arg -key value -key2 value2

#!/bin/sh

a=$(($#-1))
b=$(($#))
while [ $a -gt 0 ]; do
    eval 'key="$'$a'"; value="$'$b'"'
    echo "$key => $value"
    b=$(($b-2))
    a=$(($a-2))
done
unset a b key value

[ $(($#%2)) -ne 0 ] && echo "first_arg = $1"

Sure you can do it from the left to the right with a few changes.

This snippet code shows the key => value pairs and the first argument if it exists.

#!/bin/sh

a=$((1+$#%2))
b=$((1+$a))

[ $(($#%2)) -ne 0 ] && echo "first_arg = $1"

while [ $a -lt $# ]; do
    eval 'key="$'$a'"; value="$'$b'"'
    echo "$key => $value"
    b=$(($b+2))
    a=$(($a+2))
done

unset a b key value

Tested with 100,000 arguments, fast.

You can also iterate key => value and first optional arg from the left to the right without eval :

#!/bin/sh

a=$(($#%2))
b=0

[ $a -eq 1 ] && echo "first_arg = $1"

for value; do
    if [ $b -gt $a -a $(($b%2)) -ne $a ]; then
        echo "$key => $value"
    fi
    key="$value"
    b=$((1+$b))
done

unset a b key value

Comments

1

This also might be useful to know: you can set a value and if someone provides input, override the default with that value.

myscript.sh -f ./serverlist.txt or just ./myscript.sh (and it takes defaults)

    #!/bin/bash
    # --- set the value, if there is inputs, override the defaults.

    HOME_FOLDER="${HOME}/owned_id_checker"
    SERVER_FILE_LIST="${HOME_FOLDER}/server_list.txt"

    while [[ $# > 1 ]]
    do
    key="$1"
    shift
    
    case $key in
        -i|--inputlist)
        SERVER_FILE_LIST="$1"
        shift
        ;;
    esac
    done

    
    echo "SERVER LIST   = ${SERVER_FILE_LIST}"

Comments

1

I'm using a combination of optget and optgets to parse short and long options with or without arguments and even non-options (those without - or --):

# catch wrong options and move non-options to the end of the string
args=$(getopt -l "$opt_long" "$opt_short" "$@" 2> >(sed -e 's/^/stderr/g')) || echo -n "Error: " && echo "$args" | grep -oP "(?<=^stderr).*" && exit 1
mapfile -t args < <(xargs -n1 <<< "$(echo "$args" | sed -E "s/(--[^ ]+) /\1=/g")" )
set -- "${args[@]}"

# parse short and long options
while getopts "$opt_short-:" opt; do
  ...
done

# remove all parsed options from $@
shift $((OPTIND-1)

By that I'm able to access all options with a variable like $opt_verbose and the non-options are accessible through the default variables $1, $2, etc.:

echo "help:$opt_help"
echo "file:$opt_file"
echo "verbose:$opt_verbose"
echo "long_only:$opt_long_only"
echo "short_only:$opt_s"
echo "path:$1"
echo "mail:$2"

One of the main features is, that I'm able to pass all options and non-options in a complete random order:

#             $opt_file     $1        $2          $opt_... $opt_... $opt_...
# /demo.sh --file=file.txt /dir [email protected] -V -h --long_only=yes -s
help:1
file:file.txt
verbose:1
long_only:yes
short_only:1
path:/dir
mail:[email protected]

More details: https://stackoverflow.com/a/74275254/318765

Comments

1

The goal is to parse a command line AND create a useful usage message

code:

for arg ; do
  case "$arg" in
    --edit)   # edit file
              cd "$(dirname $0)" && vim "$0"
    ;;
    --noN)    # do NOT create 'NHI1/../tags'
              let noN=1
    ;;
    --noS)    # do NOT create 'HOME/src/*-latest/tags'
              let noS=1
    ;;
    --help)   # write this help message
    ;&
    *)        echo "usage: $(basename $0) options..." 1>&2
              awk '/--?\w+\)/' "$0"  1>&2
              exit
    ;;
  esac
done

this create the usage message:

> build_tags.bash -x
usage: build_tags.bash options...
    --edit)   # edit file
    --noN)    # do NOT create 'NHI1/../tags'
    --noS)    # do NOT create 'HOME/src/*-latest/tags'
    --help)   # write this help message

the clue is that the definition* of the case target is also the documentation of the case target.

Comments

0

Most of the time args needs to set/change some variables in script to change it's behavior. But we can do this(set/change some variables) directly without parsing args. Take a look at this script:

#!/bin/bash

a=${a:-1} # Set $a from env var $a, set default value 1 if $a is not set
b=${b:-2} # Set $b from env var $b, set default value 2 if $b is not set

echo $a $b

Lets run it:

$ ./test 
1 2

Now lets change $a:

$ a=3 ./test 
3 2

and $b:

$ a=3 b=7 ./test 
3 7

And some logic could be implemented like this:

#!/bin/bash

a=${a:-1}
b=${b:-2}

[[ $c ]] && b=

echo $a $b

check:

$ a=3 b=7 c=1 ./test 
3

But unfortunately you can't pass arrays as env vars, this won't work:

#!/bin/bash

a=${a:-1}
b=${b:-2}

[[ $c ]] && b=

echo $a $b
for i in "${data[@]}"; { echo $i; }

try:

$ a=3 b=7 data=(a b c) ./test 
3 7
(a b c)

Data must flow in like this:

#!/bin/bash

a=${a:-1}
b=${b:-2}
data=("$@")

[[ $c ]] && b=

echo $a $b
for i in "${data[@]}"; { echo $i; }

test:

$ a=3 b=7 ./test a b c
3 7
a
b
c

And a lot more...

Comments

-1

The Problem

Bash and many other similar shells like zsh, tcsh etc. suffer in this area. Even the top answers in this post do not offer an easy way to document the help of the options and the script itself.

Python's argparse does a very good job here.

The Solution

If you can afford to have an additional file dedicated to the command line arguments (which is worthwhile if you have many options and related help etc.) and if you can afford to have Python in the loop (It is installed by default on most Linux systems these days anyway), we can leverage Python's argparse in any shell scripts you might have and get the best argparser across all languages work for you.

Try this Python package https://argparse-enh.readthedocs.io/en/latest/argparse_shell.html

Disclaimer: I am the author of that Python package.

Comments

1
2

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.