How do you define control flow in a click CLI in Python based on a single option and pass the rest to another command?

Question

This question is about the Python click package and relates to control flow based on arguments passed to the CLI.

I'm trying to build a master CLI to sit at the top directory of my repo, which will control many different modes of operation from one central place. When you invoke this command, it will take perhaps n options, where the first will determine which module to invoke, and then pass along the n-1 args to another module. But I want each module's commands and options to be defined within its respective module, not the main CLI controller, as I'm trying to keep the main controller simple and each module nicely abstracted away.

Here is a simple example of what I want:

import click


@click.command()
@click.option('--foo-arg', default='asdf')
def foo(foo_arg):
    click.echo(f"Hello from foo with arg {foo_arg}")


@click.command()
@click.option('--bar-arg', default='zxcv')
def bar(bar_arg):
    click.echo(f"Hello from bar with arg {bar_arg}")


@click.command()
@click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
    if mode == "foo":
        foo()

    if mode == "bar":
        bar()


if __name__ == '__main__':
    cli()

In this example, foo() and bar() should be thought of as modules that are buried within the repo and which may also require a large number of CLI options, which I don't want to overload the main cli.py with. I feel like foo()'s CLI options should live within foo for contextual reasons. The following is how I want it to work.

Example 1:

python -m cli --mode foo --foo-arg asdf

should produce

Hello from foo with arg asdf

Example 2:

python -m cli --mode bar --bar-arg zxcv

should produce

Hello from bar with arg zxcv

Example 3:

python -m cli --mode foo --bar-arg qwer

should fail since foo() doesn't have the --bar-arg option.

Disclaimer: I know that I could register foo and bar as separate commands (invoked via python -m cli foo --foo-arg asd, i.e. with foo instead of --foo). However, due to a reason beyond the scope of this question, I need foo or bar to be specified by the --mode option identifier. This is a limitation of a tool that interacts with my app, which is unfortunately outside my control.

Is there a way to parse the args and make control flow possible based on a subset of args and then pass the remaining ones to a subsequent module, while not defining every module's options as decorators on def cli()?

Stephen Rauch · Accepted Answer · 2019-11-05 01:34:38Z

2

Using an option to invoke a subcommand can be achieved by using click.MultiCommand and a custom parse_args() method like:

Custom Class

def mode_opts_cmds(mode_opt_name, namespace=None):

    class RemoteCommandsAsModeOpts(click.MultiCommand):

        def __init__(self, *args, **kwargs):
            super(RemoteCommandsAsModeOpts, self).__init__(*args, **kwargs)
            self.mode_opt_name = '--{}'.format(mode_opt_name)
            opt = next(p for p in self.params if p.name == mode_opt_name)
            assert isinstance(opt.type, click.Choice)
            choices = set(opt.type.choices)
            self.commands = {k: v for k, v in (
                namespace or globals()).items() if k in choices}
            for command in self.commands.values():
                assert isinstance(command, click.Command)

        def parse_args(self, ctx, args):
            try:
                args.remove(self.mode_opt_name)
            except ValueError:
                pass
            super(RemoteCommandsAsModeOpts, self).parse_args(ctx, args)

        def list_commands(self, ctx):
            return sorted(self.commands)

        def get_command(self, ctx, name):
            return self.commands[name]

    return RemoteCommandsAsModeOpts

Using the Custom Class

To make use of the custom class, invoke the mode_opts_cmds() function to create the custom class and then use the cls parameter to pass that class to the click.group() decorator.

@click.group(cls=mode_opts_cmds('mode'))
@click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
    """My wonderful cli"""

How does this work?

This works because click is a well designed OO framework. The @click.group() decorator usually instantiates a click.Group object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride desired methods.

In this case we over ride click.Group.parse_args() so that when the command line is parsed we simply remove --mode and then the command line parses just like a normal sub-command.

To make this a library function

If the custom class is going to reside in a library, then instead of using the default globals() from the library file, the namespace will need to be passed in. The custom class creator method would then need to be called with something like:

@click.group(cls=mode_opts_cmds('mode', namespace=globals()))

Test Code

import click

@click.command()
@click.option('--foo-arg', default='asdf')
def foo(foo_arg):
    """the foo command is ok"""
    click.echo(f"Hello from foo with arg {foo_arg}")


@click.command()
@click.option('--bar-arg', default='zxcv')
def bar(bar_arg):
    """bar is my favorite"""
    click.echo(f"Hello from bar with arg {bar_arg}")


@click.group(cls=mode_opts_cmds('mode'))
@click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
    """My wonderful cli command"""


if __name__ == "__main__":
    commands = (
        '--mode foo --foo-arg asdf',
        '--mode bar --bar-arg zxcv',
        '--mode foo --bar-arg qwer',
        '--mode foo --help',
        '',
    )

    import sys, time

    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for command in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + command)
            time.sleep(0.1)
            cli(command.split())

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc,
                                   (click.ClickException, SystemExit)):
                raise

Test Results

Click Version: 7.0
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> --mode foo --foo-arg asdf
Hello from foo with arg asdf
-----------
> --mode bar --bar-arg zxcv
Hello from bar with arg zxcv
-----------
> --mode foo --bar-arg qwer
Usage: test.py foo [OPTIONS]
Try "test.py foo --help" for help.

Error: no such option: --bar-arg
-----------
> --mode foo --help
Usage: test.py foo [OPTIONS]

  the foo command is ok

Options:
  --foo-arg TEXT
  --help          Show this message and exit.
-----------
> 
Usage: test.py [OPTIONS] COMMAND [ARGS]...

  My wonderful cli command

Options:
  --mode [foo|bar]
  --help            Show this message and exit.

Commands:
  bar  bar is my favorite
  foo  the foo command is ok

edited Nov 5, 2019 at 1:34

answered Nov 3, 2019 at 18:56

Stephen Rauch♦

50.1k32 gold badges118 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Ben D Over a year ago

The only issue I'm seeing is when I try to move mode_opts_cmds() into a separate utils module and import it. In that case I am getting KeyError: 'foo'. Do you have any advice?

Ben D Over a year ago

I'm leaving foo and bar in the same file right next to cli(). I moved the custom class def mode_opts_cmds(mode_opt_name)... into click_utils.py and from click_utils import mode_opts_cmds at the top of cli.py. This is when it fails and produces the KeyError: 'foo' error.

Stephen Rauch Over a year ago

Sorry, you are correct, I used globals() which will be specific to the file the class definition is in. Check update for one way to handle this.

Ben D Over a year ago

Could this be modified to accept the arguments in arbitrary order, like --foo-arg asdf --mode foo? The reason I ask is that the external service that interacts with my app sorts CL arguments alphabetically, which I have no control over.

Stephen Rauch Over a year ago

@BenD, certainly. What would need to be done is instead of simply args.remove(self.mode_opt_name), you would need to keep track of where ---mode was in the list and then move the following arg to the front of the list.

paltaa · Accepted Answer · 2019-11-03 03:50:56Z

0

You could use Callbacks for validations: https://click.palletsprojects.com/en/7.x/options/#callbacks-for-validation

Also, use multi chaining commands, so each argument will be passed to the command it needs.

https://click.palletsprojects.com/en/7.x/commands/#multi-command-chaining

answered Nov 3, 2019 at 3:50

paltaa

3,2521 gold badge19 silver badges34 bronze badges

2 Comments

Ben D Over a year ago

I think the problem is that I need the logic to be driven by options, not commands. This means outside of python cli.py, everything that follows must be of format --option-name option_value. I know this is a bit of an odd requirement, but the app I'm writing is called by an external controller that can only form the command using a dictionary of option names and values, not command names.

Ben D Over a year ago

I can get my example above to work using commands such as python cli.py foo --foo-arg asdf, but not python cli.py --mode foo --foo-arg asdf (which is required).

Collectives™ on Stack Overflow

How do you define control flow in a click CLI in Python based on a single option and pass the rest to another command?

2 Answers 2

Custom Class

Using the Custom Class

How does this work?

To make this a library function

Test Code

Test Results

5 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Custom Class

Using the Custom Class

How does this work?

To make this a library function

Test Code

Test Results

5 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related