How to parse variable length command line arguments with regex?

Question

I have a large number of files that each contain a bash command with a variable amount of parameters. I need to replace these with a corresponding API call.

Example bash command in file (Note: number of '-p' arguments vary, some have none):

./some_script.sh http://some.server.com -p a=value -p b=value -p c=value

Example corresponding API call

http://some.server.com/api/some/endpoint?a=value&b=value&c=value

My issue is I can't seem to group each parameter, given that the number of parameters is variable.

Basic regex (this will match above example, but only group first parameter):

.\/some_script.sh\s([\w\/:\.]*)(\s-\w\s[\w=]*)

And I tried:

.\/some_script.sh\s([\w\/:\.]*)(\s-\w\s[\w=]*)*

However, this seems to only group the last parameter. (tested with regex101)

Ideally, I would like this regex to be able to group an indefinite number of arguments in these files so that I can easily rebuild the command as an API call.

If more detail is required please let me know, any suggestions are welcome.

user557597 · Accepted Answer · 2019-05-31 15:28:22Z

2

You need to use the \G anchor

/(?|\.\/some_script\.sh\s([\w\/:.]*)|(?!^)\G())\s-(\w)\s([\w=]*)/

https://regex101.com/r/0151qC/1

Expanded

 (?|                           # Branch reset
      \. /some_script \. sh         # First, find the script name
      \s 
      ( [\w/:.]* )                  # (1), url

   |                              # or,
      (?! ^ )
      \G                            # Start where last left off
      ( )                           # (1), blank url

 )

 \s - 
 ( \w )                        # (2), - 'p'
 \s 
 ( [\w=]* )                    # (3), 'a=value'

answered May 31, 2019 at 15:28

user557597

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2020-06-20 09:12:55Z

1

Here, maybe we could find another approach, and step by step collect our desired data from our inputs. We would then likely start with an expression similar to:

.+\.sh.+?(https?:\/\/[^\s]*)|\s+-[a-z]+\s+([\w=]+)

which has our link in here:

(https?:\/\/[^\s]*)

and our variables in:

([\w=]+)

altered with a logical OR.

We can also modify and add other boundaries or reduce our boundaries, if that might be desired.

DEMO

Test

This snippet just shows that how the capturing groups work:

const regex = /.+\.sh.+?(https?:\/\/[^\s]*)|\s+-[a-z]+\s+([\w=]+)/gm;
const str = `./some_script.sh http://some.server.com -p a=value -p b=value -p c=value
`;
let m;

while ((m = regex.exec(str)) !== null) {
    // This is necessary to avoid infinite loops with zero-width matches
    if (m.index === regex.lastIndex) {
        regex.lastIndex++;
    }
    
    // The result can be accessed through the `m`-variable.
    m.forEach((match, groupIndex) => {
        console.log(`Found match, group ${groupIndex}: ${match}`);
    });
}

RegEx Circuit

jex.im visualizes regular expressions:

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered May 31, 2019 at 15:11

Emma Marcier

27.8k12 gold badges49 silver badges71 bronze badges

1 Comment

SeeSharp Over a year ago

Perfect! Thank you very much, Emma, for the quick answer!

Collectives™ on Stack Overflow

How to parse variable length command line arguments with regex?

2 Answers 2

Comments

DEMO

Test

RegEx Circuit

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Test

RegEx Circuit

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related