Python parse string containing functions, lists and dicts

Question

I'm trying to find way to parse string that can contain variable, function, list, or dict written in python syntax separated with ",". Whitespace should be usable anywhere, so split with "," when its not inside (), [] or {}.

Example string: "variable, function1(1,3), function2([1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': "dict_item_1"}"

Another example string: "variable,function1(1, 3) , function2( [1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': "dict_item_1"}"

Example output ["variable", "function1(1,3)", "function2([1,3],2)", "['list_item_1','list_item_2']", "{'dict_key_1': "dict_item_1"}"]

edit: Reason for the code is to parse string an then run it with exec("var = &s" % list[x]). (yes i know this might not be recommended way to do stuff)

damjan · Accepted Answer · 2016-09-17 13:33:26Z

2

I guess the main problem here is that the arrays and dicts also have commas in them, so just using str.split(",") wouldn't work. One way of doing it is to parse the string one character at a time, and keep track of whether all brackets are closed. If they are, we can append the current result to an array when we come across a comma. Here's my attempt:

s = "variable, function1(1,3),function2([1,3],2),['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"

tokens = []
current = ""
open_brackets = 0

for char in s:
    current += char

    if char in "({[":
        open_brackets += 1
    elif char in ")}]":
        open_brackets -= 1
    elif (char == ",") and (open_brackets == 0):
        tokens.append(current[:-1].strip())
        current = ""

tokens.append(current)

for t in tokens:
    print(t)

"""
    variable
    function1(1,3)
    function2([1,3],2)
    ['list_item_1','list_item_2']
    {'dict_key_1': 'dict_item_1'}
"""

answered Sep 17, 2016 at 13:33

damjan

761 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

chapelo Over a year ago

I was thinking of the same idea using a list as a stack of brackets, but the open_brackets counter works the same and is simpler.

SacredCoconut Over a year ago

Yeah i just thought that regex/python would have had way of doing it instead of writing algorithm my self. I will have to do that then i suppose.

ShadowRanger · Accepted Answer · 2016-09-17 13:08:50Z

0

Regular expressions aren't very good for parsing the complexity of arbitrary code. What exactly are you trying to accomplish? You can (unsafely) use eval to just evaluate the string as code. Or if you're trying to understand it without evaling it, you can use the ast or dis modules for various forms of inspection.

answered Sep 17, 2016 at 13:08

ShadowRanger

158k12 gold badges222 silver badges317 bronze badges

6 Comments

SacredCoconut Over a year ago

Reason for the code is to parse string into list and then run it with exec(var = list[x]).

ShadowRanger Over a year ago

@SacredCoconut: Is there a reason not to bulk parse it all at once?

SacredCoconut Over a year ago

Sorry i don't know what you mean by bulk parse.

ShadowRanger Over a year ago

@SacredCoconut: As in, why not just do allvals = eval(thestring), then index the resulting tuple for the results?

SacredCoconut Over a year ago

Oh that might work, i have most likely misunderstood how eval works

|

Jack Evans · Accepted Answer · 2016-09-17 12:57:46Z

0

Have you tried using split?

>>> teststring = "variable, function1(1,3), function2([1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"
>>> teststring.split(", ")
['variable', 'function1(1,3)', 'function2([1,3],2)', "['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"]

answered Sep 17, 2016 at 12:57

Jack Evans

1,7173 gold badges18 silver badges36 bronze badges

1 Comment

SacredCoconut Over a year ago

Oh yeah i forgot to mention it might or might not have whitespace after ",". For example "['variable', function1(1, 3)'" would not work.

Collectives™ on Stack Overflow

Python parse string containing functions, lists and dicts

3 Answers 3

2 Comments

6 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

6 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related