Split string that looks like a Python function call to arguments

Question

I am trying to make a function that accepts string that looks like function call in python and returns the arguments to the function Example:

"fun(1, bar(x+17, 1), arr = 's,y')"

will result:

["1", "bar(x+17, 1)", "arr = 's,y'"]

The problem of using regular expressions is that I don't know if it is possible to not split at the commas inside parenthesis or quotes. Thanks.

Edit: this Python: splitting a function and arguments doesn't answer correctly the quastions since it doesn't treat commas in parenthesis or quotes.

As @Kevin said, regular expressions cannot solve this since they can't handle nested parenthesis.

possible duplicate of Regex question about parsing method signature — vaultah
– vaultah, Commented Jul 21, 2015 at 18:36
possible duplicate of Regular expression to return text between parenthesis — letsc
– letsc, Commented Jul 21, 2015 at 18:36
You should use a regular expression to parse the stuff in between parentheses, and then split that that string to find your arguments — Jeremy Fisher
– Jeremy Fisher, Commented Jul 21, 2015 at 18:36
All these do not treat correctly commas in parenthesis or quotes. like in the example — tal
– tal, Commented Jul 21, 2015 at 18:37
"vanilla" regexes can't parse nested parentheses. Maybe you can do it with more advanced features, but at some point it's going to be complex enough that you may as well just write a parser. — Kevin
– Kevin, Commented Jul 21, 2015 at 18:43

Joran Beasley · Accepted Answer · 2015-07-21 18:48:01Z

3

you can keep track of your own state fairly simply with something like

def parse_arguments(s):
    openers = "{[\"'("
    closers = "}]\"')"
    state = []
    current = ""
    for c in s:
        if c == "," and not state:
           yield current
           current = ""
        else:
           current += c
           if c in openers:
              state.append(c)
           elif c in closers:
              assert state, "ERROR No Opener for %s"%c
              assert state[-1] == openers[closers.index(c)],"ERROR Mismatched %s %s"%(state[-1],c)
              state.pop(-1)
    assert not state, "ERROR Unexpected End, expected %s"%state[-1]
    yield current

print list(parse_arguments("1, bar(x+17, 1), arr = 's,y'"))

answered Jul 21, 2015 at 18:48

Joran Beasley

114k13 gold badges167 silver badges187 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user2124834 Over a year ago

Doesn't seem to work for parse_arguments("hello"), I think because " is both closing and opening? I think it also will try to match openers/closers within strings, which it shouldn't.

user2124834 Over a year ago

I tried my best at an enhancement: stackoverflow.com/a/61633243/2124834

Avinash Raj · Accepted Answer · 2015-07-21 18:57:06Z

2

Give a try to this complex split function.

>>> import re
>>> s = "fun(1, bar(x+17, 1), arr = 's,y')"
>>> [i.strip() for i in re.split(r'''^\w+\(|\)$|((?:\([^()]*\)|'[^']*'|"[^"]*"|[^'"(),])*)''', s) if i and i !=',']
['1', 'bar(x+17, 1)', "arr = 's,y'"]

answered Jul 21, 2015 at 18:57

Avinash Raj

175k32 gold badges247 silver badges289 bronze badges

2 Comments

Joran Beasley Over a year ago

good answer ... but im pretty sure for any regex there exists some counter example that will break it .. (with nested parenthesis) +1 all the same as I cant think up a good counter example offhand (and even if i could) ... but debugging it when it doesnt work might be a little painful

Joran Beasley Over a year ago

that said I think there is an actual regexp module in pipi import regex that does support stack memory(and other advanced regex features)and so you could come up with a perfect regex solution using that module

RemcoGerlich · Accepted Answer · 2015-07-21 18:53:16Z

1

It would be nice to do the with the ast (abstract syntax tree) standard library module, although it might be overkill:

>>> import ast
>>> parsed = ast.parse("fun(1, bar(x+17, 1), arr='s, y')")
>>> ast.dump(p.body[0].value)
"Call(func=Name(id='fun', ctx=Load()), args=[Num(n=1), 
Call(func=Name(id='bar', ctx=Load()), args=[BinOp(left=Name(id='x', 
ctx=Load()), op=Add(), right=Num(n=17)), Num(n=1)], keywords=[], 
starargs=None, kwargs=None)], keywords=[keyword(arg='arr', 
value=Str(s='s, y'))], starargs=None, kwargs=None)"

Unfortunately there's no standard library way to get those back to standard strings like "1", "bar(x+17, 1)" and "arr='s, y'". But https://pypi.python.org/pypi/astor can probably do that.

answered Jul 21, 2015 at 18:53

RemcoGerlich

31.4k6 gold badges66 silver badges83 bronze badges

Comments

vks · Accepted Answer · 2015-07-21 19:06:15Z

1

import re
x="fun(1, bar(x+17, 1), arr = 's,y')"
print re.split(r",\s*(?![^\(]*\))(?![^']*'(?:[^']*'[^']*')*[^']*$)",re.findall(r"^.*?\((.*)\)",x)[0])

You can try using re.

Output:['1', 'bar(x+17, 1)', "arr = 's,y'"]

answered Jul 21, 2015 at 19:06

vks

68.1k11 gold badges96 silver badges132 bronze badges

1 Comment

tal Over a year ago

as all regex answers, they fail on nested parenthesis. try x = "fun(bar(1,foo(1)))"

user2124834 · Accepted Answer · 2020-05-06 10:47:45Z

Based on Joran Beasley's answer with hopefully better string handling? The only change is the new if-arm, allowing any characters when we're in a string, including an escaped quote.

def parse_arguments(s):
    openers = "{[\"'("
    closers = "}]\"')"
    state = []
    current = ""
    for c in s:
        if c == "," and not state:
            yield current
            current = ""
        else:
            current += c
            if state and state[-1] in "\"'":
                if c == state[-1] and current[-1] != "\\":
                    state.pop(-1)
            else:
                if c in openers:
                    state.append(c)
                elif c in closers:
                    assert state, "ERROR No Opener for %s" % c
                    assert (
                        state[-1] == openers[closers.index(c)]
                    ), "ERROR Mismatched %s %s" % (state[-1], c)
                    state.pop(-1)
    assert not state, "ERROR Unexpected End, expected %s" % state[-1]
    yield current

alwaysmpe · Accepted Answer · 2025-01-16 00:21:59Z

The regex package supports recursive regular expressions so the below (rather nasty) regex gets most of the way there (it doesn't support keyword arguments, possibly something else I've missed)

import regex
arg_cap = r''' *(\w+) *'''
qote_cap = r''' *((?P<qseq>(?P<qch>["'])(?:(?P=qch){2})?).*?(?<!\\)(?P=qseq)) *'''
call_cap = r''' *\w+\((?: *(?&expr)( *,? *(?&expr))*)\)'''
num_cap = r''' *(\d*) *'''
seq_cap = r''' *(?:(?P<sd>\{)|(?P<lst>\[)|(?P<tp>\()) *(?P<kv>(?&expr) *(?:: *(?&expr))? *(?:, *(?&kv) *)*,? *)(?(lst)\])(?(tp)\))(?(sd)\}) *'''
fn_break = fr'''^\w+\((?:(?P<arg>(?P<expr>{arg_cap}|{qote_cap}|{call_cap}|{num_cap}|{seq_cap}))(?:, *(?&arg))*)\)$'''

fn_str = """fun(1, xyz, foo(a, b), (a, b, ((d, e,))), "xbc", '''trip quote''', {set, values}, {abc: def, ghi: jkl})"""
mat = regex.match(fn_break, fn_str)

assert mat is not None
print(mat.captures("arg"))

outputs

['1', 'xyz', 'foo(a, b)', '(a, b, ((d, e,)))', '"xbc"', "'''trip quote'''", '{set, values}', '{abc: def, ghi: jkl}']

That said, you're probably better off using an ast parser such as the standard library ast or pylint's astroid, because I can barely understand the above code, and I wrote it.

_{_{(I may have spent too much time writing regular expressions...)}}

Collectives™ on Stack Overflow

Split string that looks like a Python function call to arguments

6 Answers 6

2 Comments

2 Comments

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

2 Comments

2 Comments

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related