Does ArgumentParser support different arguments per file, ffmpeg style?

Question

I want my application to work on several files and have a different set of options for each input file:

python my.py -a file_a -b file_b --do-stuff=x file_c

ffmpeg uses this idea for its command line arguments.

I tried the following:

parser = argparse.ArgumentParser()
parser.add_argument("-a", action="store_true")
parser.add_argument("-b", action="store_true")
parser.add_argument("--do-stuff", type=str)
parser.add_argument("files", type=pathlib.Path, nargs="+")
args = parser.parse_args()

And I get an error:

error: unrecognized arguments: file_b file_c

Can I use ArgumentParser to parse such a command line?

The goal is to get a mapping like this:

file_a: {a: True}
file_b: {b: True}
file_c: {do_stuff: 'x'}

docs.python.org/3/library/argparse.html#intermixed-parsing is as close as it gets — jonrsharpe
– jonrsharpe, Commented Jul 15 at 9:29
You define -a and -b as flags which are either set or not set, they do not take a value. Hence file_a and file_b don't belong to anything and aren't recognised as anything. Does that explain the problem? What else did you want this to mean? Assume we do not know of the top off our heads how ffmpeg would treat this… — deceze
– deceze ♦, Commented Jul 15 at 10:20

anatolyg · Accepted Answer · 2025-07-15 18:40:57Z

No, ArgumentParser doesn't support having different options for different files. It doesn't take the order of options (relative to positional arguments) into account.

As noted by jonrsharpe, https://docs.python.org/3/library/argparse.html#intermixed-parsing is as close as it gets. And it is not good enough, because a call to parse_known_intermixed_args or parse_intermixed_args will extract all options, regardless of their position relative to file_a.

It's possible to "prepare" arguments for the parser and parse them for each file separately. First, collect file names (or "positional arguments" as they are called) from the command line. For each file name, extract arguments which precede it. Then parse these arguments.

# Change the type to str so each file name could be found later
parser.add_argument("files", type=str, nargs="+")

# Collect file names; ignore all options
files = parser.parse_intermixed_args().files

args_per_file = {}
args_left = sys.argv[1:]
for file in files:
    # Find the file name (guaranteed to succeed)
    p = args_left.index(file) + 1
    # Parse only the arguments which are relevant to this file
    args_per_file[pathlib.Path(file)] = parser.parse_args(args_left[:p])
    # Delete what was parsed
    del args_left[:p]

Dunes · Accepted Answer · 2025-07-15 20:39:35Z

You can get close to parsing what you want, but you would need to apply a transformation on the resulting namespace returned by the parser. The two differences you would need to make is that instead of --do-stuff=x file_c you have --do-stuff x file_c (NB. no equals sign is allowed). And you would need to apply some post processing to verify x and file_c were valid arguments to --do-stuff.

Example:

import argparse
from collections import defaultdict
from pathlib import Path

def valid_file(name: str) -> str:
    """Checks if a filename exists"""
    path = Path(name)
    if not path.exists():
        raise argparse.ArgumentTypeError(f'{name} does not exist')
    return name  # can conver to another type if desired

# use the "append" action to store all uses of each option
parser = argparse.ArgumentParser()
parser.add_argument("-a", type=valid_file, action='append')
parser.add_argument("-b", type=valid_file, action='append')
# use nargs=2 to enforce that --do-stuff must always have two arguments
parser.add_argument("--do-stuff", type=str, nargs=2, action='append')

# parsing example usage
ns = parser.parse_args(
    '-a file_a -b file_b -a file_c --do-stuff x file_d --do-stuff y file_e'
    .split()
)
assert vars(ns) == {
    'a': ['file_a', 'file_c'],
    'b': ['file_b'],
    'do_stuff': [
        ['x', 'file_d'],
        ['y', 'file_e'],
    ],
}

# verify do_stuff usage that cannot easily be done with argparse.
# ie. enforce first argument is a valid option and second argument is a valid file
for first, second in ns.do_stuff:
    if first not in ('x', 'y', 'z'):
        print('invalid option to do_stuff')
        parser.print_help()
        exit(1)
    if not Path(second).exists():
        print(f'do_stuff error: {second} does not exist')
        parser.print_help()
        exit(1)

# convert argparse namespace into the desired dictionary
# uses a defaultdict(list) so that it is possible to spot if some files received
# multiple options. eg. -a foo --do-stuff x foo
result = defaultdict(list)
for option, values in vars(ns).items():
    for value in values:
        if isinstance(value, list):
            assert len(value) == 2
            result[value[1]].append({option: value[0]})
        else:
            result[value].append({option: True})

# optional final validation -- enforce that each file received at most one option
too_many_options = [file_ for file_, options in result.items() if len(options) > 1]
if too_many_options:
    print('the following input files received multiple options. Only 1 is allowed')
    print(too_many_options)
    parser.print_help()
    exit(1)

assert result == {
    'file_a': [{'a': True}],
    'file_b': [{'b': True}],
    'file_c': [{'a': True}],
    'file_d': [{'do_stuff': 'x'}],
    'file_e': [{'do_stuff': 'y'}],
}

Collectives™ on Stack Overflow

Does ArgumentParser support different arguments per file, ffmpeg style?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related