Get dictionary from csv file by conversion

Question

I'm trying to convert a csv to a dict, but I can't get the right output.

Basically I have a csv file which contains integers in the 6th and 7th column, which should be converted in 'lat' and 'lon' in the dict.

This is what I'm trying:

with open('secondhand_data.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=';')
    output2['lon'] = {row[6] for row in reader}
    output2['lat'] = {row[7] for row in reader}

My result is:

{'lat': set([]), 'lon': set(['', '16.33052', '16.38002', '16.38414', '16.34794', '16.34797', '16.33351', '16.46922', '16.33353', '16.33367', '16.38552', '16.33354', '16.41716', ...])}

But what I need is precisely:

{'lat': ['53.1445116550943', '53.134787053494', '53.1383785260816', '53.330366', '55.8611098159417', ...], 'lon' : ['', '16.33052', '16.38002', '16.38414', '16.34794', '16.34797', '16.33351', ...]}

Does anyone know what I'm doing wrong? Many thanks in advance!

@jacoblauw: the csv file looks like this... its a bit messy, sorry for that.

Waage, Deko/ Vintage/ Retro/ Nostalgie;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/waage-deko-vintage-retro-nostalgie-208813335/;"
Doris S.";10;1030 Wien, 03. Bezirk, Landstraße; ;16.38575;48.19149;Fasangasse, Ecke Rennweg1030 Wien, 03. Bezirk, LandstraßeWien
schloss;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/schloss-208789390/;"
Angelina";10;1100 Wien, 10. Bezirk, Favoriten; ;16.35809;48.17209;Hardtmuthgasse1100 Wien, 10. Bezirk, FavoritenWien
RC Boot;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/rc-boot-208786703/;"
privat";15;1100 Wien, 10. Bezirk, Favoriten; ;;;
taschenmesser solingen dirlam & Sohn Söhne horn kleine beschädigung mit hülle;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/taschenmesser-solingen-dirlam-sohn-soehne-horn-kleine-beschaedigung-mit-huelle-208768953/;"
rabe";25;1200 Wien, 20. Bezirk, Brigittenau; ;16.37349;48.23059;brigittenau1200 Wien, 20. Bezirk, BrigittenauWien
Konvolut Gama Norev Matchbox Lesney Corgi Wiking Schuco piccolo...14 teilig, alt, defekt;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/konvolut-gama-norev-matchbox-lesney-corgi-wiking-schuco-piccolo-14-teilig-alt-defekt-208752031/;"
Peter";50;1190 Wien, 19. Bezirk, Döbling; ;16.35537;48.27574;Heiligenstädterstr.1190 Wien, 19. Bezirk, DöblingWien
DAS HAUS DER SCHWÄNE/ A. J. CRONIN, geb. Jubiläumsausgabe von 1934,;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/das-haus-der-schwaene-a-j-cronin-geb-jubilaeumsausgabe-von-1934-208735636/;"
Annemarie";5;1230 Wien, 23. Bezirk, Liesing; ;16.28017;48.15508;Karl Schwed Gasse1230 Wien, 23. Bezirk, LiesingWien
Röhrenradio;http://www.willhaben.at/iad/kaufen-und-verkaufen/d/roehrenradio-208710345/;"

Can you post the csv, or a small amount that can be used to reproduce your results? — jacoblaw
– jacoblaw, Commented Jun 23, 2017 at 16:56
your reader is completely consumed by the output2['lon'] = {row[6] for row in reader} line. there is nothing left for lat. — stefan
– stefan, Commented Jun 23, 2017 at 17:00
@ stefan, sorry it wasn't clear, but of course I took that in account. — fahrradlaus
– fahrradlaus, Commented Jun 23, 2017 at 17:05

cs95 · Accepted Answer · 2017-06-23 17:25:48Z

2

Replace your set comprehension with a list comprehension. Also, your data is not consistent. To handle this, you should place your code in a try-except brace and also check for non-empty values.

import csv
with open('...', 'r') as f:
    reader = csv.reader(f, delimiter=';')
    output2 = { 'lat' : [], 'lon' : [] }
    for row in reader:
        try:
            if row[7] != '' and row[6] != '': 
                output2['lat'].append(row[7])
                output2['lon'].append(row[6])
        except:
            pass
print(output2)

Output:

{'lat': ['48.17209', '48.23059', '48.27574', '48.15508'], 'lon': ['16.35809', '16.37349', '16.35537', '16.28017']}

edited Jun 23, 2017 at 17:25

answered Jun 23, 2017 at 16:56

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

fahrradlaus Over a year ago

Apart from the fact that row[6] and row[7] are still right, see .csv file edit, this still gives me: {'lat': set(['', '48.1803', '48.20549', ...], 'lon': set(['', '16.33052', '16.38002', ...])} and not {'lat': [...], 'lon': [...]}

stefan Over a year ago

downvoted because of a careless answer. the idices are not off by one as as he gets logitude right. to make it worse you extend your answer with claiming an iterator beeing consumed twice.

developer_hatch Over a year ago

I think your answer is ok, plus one for care in editing

cs95 Over a year ago

@DamianLattenero Thank you. I take my work seriously on this site. While the bad comments won't go away, I can at least do right by myself in not providing misinformation.

developer_hatch Over a year ago

@stefan I don't follow, what's wrong with Coldspeed answer? Exactly what line, and why? Because "your reader object is completely consumed" and "to make it worse you extend your answer with claiming an iterator beeing consumed twice" doensn't helps really... Don't take it wrong

|

developer_hatch · Accepted Answer · 2017-06-23 17:31:36Z

2

You're using set comprehension and attaching the set to a key. The problem with this is three fold:

1) You're stepping through your reader twice.

2) Sets are unordered.

3) Sets only store 1 copy of any value

Try this:

output2 = {'lat': [], 'lon': []}
with open('secondhand_data.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=';')
    for row in reader:
        output2['lat'].append(row[7])
        output2['lon'].append(row[6])

edited Jun 23, 2017 at 17:31

developer_hatch

16.3k3 gold badges47 silver badges79 bronze badges

answered Jun 23, 2017 at 17:11

Alan Leuthard

8086 silver badges11 bronze badges

Comments

developer_hatch · Accepted Answer · 2017-06-23 18:06:00Z

2

You can do it simply like:

dct = {'lat': [], 'lon': []}
with open('secondhand_data.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=';')
    for row in reader:
        dct['lat'] += list(row[7])
        dct['lon'] += list(row[6])

edited Jun 23, 2017 at 18:06

answered Jun 23, 2017 at 17:22

developer_hatch

16.3k3 gold badges47 silver badges79 bronze badges

4 Comments

Alan Leuthard Over a year ago

Wouldn't this simply result in each key containing a list that contains the column values of only the final row?

developer_hatch Over a year ago

@AlanLeuthard yes you're right! I will edit right now, my bad haha

Alan Leuthard Over a year ago

So which is a more efficient way to add to a list? Using the + operator, or using append/extend?

developer_hatch Over a year ago

@AlanLeuthard I searched, append is twice faster, :) see here, stackoverflow.com/questions/725782/…, I think now your answer is better haha, plus one for that

Collectives™ on Stack Overflow

Get dictionary from csv file by conversion

3 Answers 3

8 Comments

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related