python csv to dictionary using csv or pandas module

Question

I am using Python's csv.DictReader to read in values from a CSV file to create a dictionary where keys are first row or headers in the CSV and other rows are values. It works perfectly as expected and I am able to get a dictionary, but I only want certain keys to be in the dictionary rather than all of the column values. What is the best way to do this? I tried using csv.reader but I don't think it has this functionality. Maybe this can be achieved using pandas?

Here is the code I was using with CSV module where Fieldnames was the keys that I wanted to retain in my dict. I realized it isn't used for what I described above.

import csv
with open(target_path+target_file) as csvfile:
    reader = csv.DictReader(csvfile,fieldnames=Fieldnames)
    for i in reader:
        print i

try to use "pandas.read_csv" method with parameter "usecols" — Konstantin Purtov
– Konstantin Purtov, Commented Jun 4, 2017 at 18:28

Onel Harrison · Accepted Answer · 2017-06-04 18:41:32Z

14

You can do this very simply using pandas.

import pandas as pd

# get only the columns you want from the csv file
df = pd.read_csv(target_path + target_file, usecols=['Column Name1', 'Column Name2'])
result = df.to_dict(orient='records')

Sources:

edited Jun 4, 2017 at 18:41

answered Jun 4, 2017 at 18:28

Onel Harrison

1,33412 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

anekix Over a year ago

thanks for reply but is the end result a list of dicts? i wanted to convert each row into a dict with with some keys. let me try this . thanks :)

Onel Harrison Over a year ago

It didn't before, but the updated answer should work for you once you change the column names to be what you have in your data.

anekix Over a year ago

can i also rename some of the existing key names like i provide a mapping between old keys and new keys?

Onel Harrison Over a year ago

Here's a SOA for renaming existing key names stackoverflow.com/questions/11346283/…

anekix Over a year ago

thanks again just one last thing there are some columns with empty values in csv file so in dictionary it become nan .is there any way to make hem default to 0 for empty cells in csv. should i ask a seperate question for it?

|

sirfz · Accepted Answer · 2017-06-04 18:35:26Z

5

You can use the to_dict method to get a list of dicts:

import pandas as pd

df = pd.read_csv(target_path+target_file, names=Fieldnames)

records = df.to_dict(orient='records')

for row in records:
    print row

to_dict documentation:

In [67]: df.to_dict?
Signature: df.to_dict(orient='dict')
Docstring:
Convert DataFrame to dictionary.

Parameters
----------
orient : str {'dict', 'list', 'series', 'split', 'records', 'index'}
    Determines the type of the values of the dictionary.

    - dict (default) : dict like {column -> {index -> value}}
    - list : dict like {column -> [values]}
    - series : dict like {column -> Series(values)}
    - split : dict like
      {index -> [index], columns -> [columns], data -> [values]}
    - records : list like
      [{column -> value}, ... , {column -> value}]
    - index : dict like {index -> {column -> value}}

      .. versionadded:: 0.17.0

    Abbreviations are allowed. `s` indicates `series` and `sp`
    indicates `split`.

Returns
-------
result : dict like {column -> {index -> value}}
File:      /usr/local/lib/python2.7/dist-packages/pandas/core/frame.py
Type:      instancemethod

answered Jun 4, 2017 at 18:35

sirfz

4,29727 silver badges38 bronze badges

5 Comments

anekix Over a year ago

thanks a lot for your reply wish i could accept more than one answer.but can you please tell if there is any way that i can rename some existing keys by providing a mapping or something?

sirfz Over a year ago

The names attribute allows you to provide your own column names. If your file contains a header, just include header=0 (i.e. 1st line) to make sure you replace the header with your own mappings in names. In addition, if you wish to only keep specific columns, you can specify them in the usecols parameter.

anekix Over a year ago

i want to keep specefic columns but after that rename those columns

sirfz Over a year ago

That's exactly what the names parameter allows you to do. Refer to pandas.read_csv documentation for details and play around with it to get a better grasp of its features.

anekix Over a year ago

looking into it

Collectives™ on Stack Overflow

python csv to dictionary using csv or pandas module

2 Answers 2

8 Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related