Setting metaflow decorator using command line parameter

Question

I have a metaflow flowspec with a decorator @some_decorator(parameter="something"). Is it possible to set things up so that I can set the parameter to the decorator via a command line parameter when I run the flow?

I tried a couple of approaches, but reached dead ends.

metaflow.Parameter won't work because the parameter only becomes available once we've entered the flow, whereas I need when the decorator is called.
I tried grabbing the parameter with argparse. But it turns out the flow file gets executed several times by metaflow, with different arguments, so this did not work.

Please provide enough code so others can better understand or reproduce the problem. — Community
– Community Bot, Commented Jan 8 at 14:43

Savin · Accepted Answer · 2025-01-29 02:54:55Z

1

Yes, you can rely on config support within Metaflow to pass these values from the command line. Here are the official docs for this - https://docs.metaflow.org/metaflow/configuring-flows/basic-configuration#defining-config-files

answered Jan 29 at 2:54

Savin

1916 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Foster Jan 29 at 4:29

Thank you, this is exactly what I was looking for. Unfortunately, the codebase I am working in is not compatible with metaflow>=2.12.37, which is required for config support (it looks like this functionality was added quite recently). Accepting the answer for the sake of anyone else who comes across this question, though.

Savin Jan 30 at 5:03

One of the goals of metaflow is to be strongly backward compatible. I am curious: what's blocking the move from 2.12.37?

Foster Jan 30 at 19:21

My company has a library with some extensions to metaflow (we use for some custom flow/step decorators). We've been using metaflow=2.12.27, and starting with 2.12.29 there was a move metaflow.metadata -> metaflow.metadata_provider that broke some of our imports.

Collectives™ on Stack Overflow

Setting metaflow decorator using command line parameter

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related