How to convert string containing list of dictionaries to python (object) LIST of DICT

Question

Using python socket I am receiving value as a string and want to store the value in SQLite DB as dictionary key to column and values as a row.

So for that purpose, I am not able to convert the string to dictionary -

ReceivedValue = [{'Value1': 'OutPut00', 'Value2': 'OutPut01', 'Value3': '2253.23'},{'Value1': 'NA', 'Value3': 'NA'}]

ReceivedValue always returns str type instead list and stopping me to process further to add in SQLite DB.

How can I convert this received value to a Dictionary or list of dictionaries?

Json Approach EDIT:1 - Attached the required screenshot for Json Error

eval Approach EDIT:2 -

def process(ReceivedValue):
     print ReceivedValue
     finVal = ast.literal_eval(ReceivedValue)
     print finVal
     print type(finVal)

OutPut -

"[{'APACHE_MODJK': '1.2.41', 'PROCESSOR_MODEL': ' Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz', 'APACHE_HOME': '/opt/www/apache/2.4.17', 'OS_KERNEL': '2.6.32-573.26.1.el6.x86_64', 'APACHE_SSO': 'YES', 'TOMCAT_VER': 'NA', 'ENV': 'PROD', 'APACHE': 'RUNNING', 'JBOSS_HOME': '/opt/www/jboss/4.3/jboss-as', 'APACHE_VIP': 'NA', 'CPU': 2, 'OS_VER': 'Red Hat Enterprise Linux Server release 6.7 (Santiago)', 'TOMCAT_DB': 'NA', 'SAN': 'NA', 'JBOSS_DB': 'p377', 'MEMORY': 4, 'JBOSS_VER': '4.3.0.GA_CP02', 'JDK_VER': '1.6.0_18', 'APACHE_VER': '2.4.17', 'MACHINE_TYPE': 'VMware Virtual Platform'}]"
[{'APACHE_MODJK': '1.2.41', 'PROCESSOR_MODEL': ' Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz', 'APACHE_HOME': '/opt/www/apache/2.4.17', 'OS_KERNEL': '2.6.32-573.26.1.el6.x86_64', 'APACHE_SSO': 'YES', 'TOMCAT_VER': 'NA', 'ENV': 'PROD', 'APACHE': 'RUNNING', 'JBOSS_HOME': '/opt/www/jboss/4.3/jboss-as', 'APACHE_VIP': 'NA', 'CPU': 2, 'OS_VER': 'Red Hat Enterprise Linux Server release 6.7 (Santiago)', 'TOMCAT_DB': 'NA', 'SAN': 'NA', 'MEMORY': 4, 'JBOSS_VER': '4.3.0.GA_CP02', 'JDK_VER': '1.6.0_18', 'APACHE_VER': '2.4.17', 'MACHINE_TYPE': 'VMware Virtual Platform'}]
<type 'str'>

WORKING SOLUTION: Just Removed the double quotes from the RecievedValue to evaluate it across eval() or ast.literal_eval()

you could throw the string into the eval function, but that's a potential security issue — Grisha
– Grisha, Commented Nov 20, 2017 at 20:26
no need for eval, ast.literal_eval does the job and is safe. — Jean-François Fabre
– Jean-François Fabre ♦, Commented Nov 20, 2017 at 20:37

Vipin Joshi · Accepted Answer · 2017-11-21 05:56:33Z

2

Use json module to convert it to dict, that would be easiest I guess.

import json
your_dict = json.loads([{'Value1': 'OutPut00', 'Value2': 'OutPut01', 'Value3': '2253.23'},{'Value1': 'NA', 'Value3': 'NA'}])

answered Nov 21, 2017 at 5:56

Vipin Joshi

3125 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rahul Trivedi Over a year ago

earlier I tried but resulted in "ValueError: No JSON object could be decoded"

Vipin Joshi Over a year ago

Paste s screenshot of your received value and it's type() here ?

Rahul Trivedi Over a year ago

Updated the screenshot in question update - RecievedValue is populated from the socket as string.

Greg Mueller · Accepted Answer · 2017-11-21 14:05:49Z

0

Assuming ReceivedValue is a string like below (your example did not have the quotes around it making it a list of dictionaries) you can use eval(ReceivedValue) to convert it.

[Standard Disclaimer applies] Note that there are security implications to using eval - do you trust the source of the string you are about to evaluate? Evaluating malicious string could cause it to execute code on your system. https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html

>>> ReceivedValue = "[{'Value1': 'OutPut00', 'Value2': 'OutPut01', 'Value3': '2253.23'},{'Value1': 'NA', 'Value3': 'NA'}]"
>>> 
>>> my_list = eval(ReceivedValue)
>>>
>>> my_list
[{'Value1': 'OutPut00', 'Value2': 'OutPut01', 'Value3': '2253.23'}, {'Value1': 'NA', 'Value3': 'NA'}]
>>>
>>> my_list[0]
{'Value1': 'OutPut00', 'Value2': 'OutPut01', 'Value3': '2253.23'}
>>>
>>> my_list[1]
{'Value1': 'NA', 'Value3': 'NA'}
>>>

answered Nov 21, 2017 at 14:05

Greg Mueller

5264 silver badges8 bronze badges

2 Comments

Rahul Trivedi Over a year ago

Thanks, @Gred Mueller for the response and I see eval and ast.literal_eval are working when I try code line by line from console but when I use them in script they always return type str, I have updated the eval approach in question as well

Rahul Trivedi Over a year ago

So finally I got the code working by removing the double quote from the ReceivedValue -- Thanks, but I am using ast.literal_eval().

James Russo · Accepted Answer · 2017-11-20 20:32:54Z

0

When working with sockets you have to serialize and deserialize information if you want to send anything besides bytes or strings across a socket. This can be accomplished in multiple ways. One such popular way is using the Pickle library https://docs.python.org/3/library/pickle.html which allows you to serialize and deserialize information such as dictionaries and objects. This can be accomplished with the dump and load functions. Just call serialized_dict = pickle.dump(dict) before sending the information in the socket and deserialized_dict = pickle.load(buffer_bytes) to deserialize it on the other end.

However it is known that the pickle function is not secure against erroneous or maliciously constructed data. And it is not advised to use pickling for data that you do not construct yourself such as user input. Another way to handle serializing is to pack and unpack buffers on your own using the struct object in python https://docs.python.org/3/library/struct.html. This is a slightly more complex approach. Here is a synopsis I can elaborate on if need be. You would need to pack each key and value pair into a buffer/bytearray and then send it through the socket, once received you would need to unpack each key and value pair then construct your dictionary again on the other side. Things you would need to make sure you keep track of are the size of each element in the buffer and how many items(key/value pairs) are in the buffer

edited Nov 20, 2017 at 20:32

answered Nov 20, 2017 at 20:26

James Russo

5584 silver badges19 bronze badges

2 Comments

Rahul Trivedi Over a year ago

Thanks for the response, although I tried to pickle.dump the final list from the client but ended with EOF error at server receiver end, and because everything is working as expected except storing data in SQLite, I don't want to manipulate client or server-side script. If I could convert received string into a dictionary then it would be easy to process all into SQLite.

Rahul Trivedi Over a year ago

Thanks for your lead, Now I am serializing data packets using struct to pack and unpack with the reference you gave link it was really helpful and better way to handle data between server and client socket.

Collectives™ on Stack Overflow

How to convert string containing list of dictionaries to python (object) LIST of DICT

3 Answers 3

3 Comments

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related