Append to a 2 dimensional array with mixed while reading a csv file in python

Question

I have a comma seperated csv file "Brent 3.csv": Rows look like this:

2014.03.12 23:59:59,2014.03.20 23:59:59,BRENTSPOT,Brent 
1,1.29,1.6,0.8568833439015613,91.09,3.5,2.053,-0.035\n

2014.04.01 23:59:59,2014.04.02 23:59:59,BRENTSPOT,Brent 
1,1.39,1.4,0.8568833439015613,89.59999999999999,3.5,2.053,-0.036\n

Now what I wanted to do is to create a two dimensional mixed list with strings and float values.

import os

def create_list(stratname,directory):
    os.chdir(directory)
    temp = []

    for file in glob.glob("*.csv"):
        if stratname in file:
            TDFile=open(file,"r")

            for i,line in enumerate(TDFile):
                s = line.split(',')
                for x in s:   
                    try:
                        temp.append(float(x))
                    except ValueError:
                        temp.append(x)
    return temp


brent3 = create_list("Brent 3",strategydir)

print(brent3)

I know that I should use

temp.append([float(x)])

but that does only create a list that looks like that:

['2014.03.12 23:59:59', '2014.03.20 23:59:59', 'BRENTSPOT', 'Brent 1', [1.29], [1.6], [0.8568833439015613], [91.09], [3.5], [2.053], [-0.035], '2014.04.01 23:59:59',    ...   ]

It should be:

[['2014.03.12 23:59:59', '2014.03.20 23:59:59', 'BRENTSPOT', 'Brent 1', 1.29, 1.6, 0.8568833439015613, 91.09, 3.5, 2.053, -0.035], ['2014.04.01 23:59:59', .....]]

I just can't seem to find the answer how to get each line of the csv file in [[first line],[second line]] format.

Any help appreciated :)

tobiasraabe · Accepted Answer · 2018-07-24 09:05:36Z

1

Why not use pandas:

import io
import pandas as pd

csv = """2014.03.12 23:59:59,2014.03.20 23:59:59,BRENTSPOT,Brent 1,1.29,1.6,0.8568833439015613,91.09,3.5,2.053,-0.035\n
2014.04.01 23:59:59,2014.04.02 23:59:59,BRENTSPOT,Brent 1,1.39,1.4,0.8568833439015613,89.59999999999999,3.5,2.053,-0.036\n"""

print(pd.read_csv(io.StringIO(csv), header=None).values.tolist())

which yields

[['2014.03.12 23:59:59', '2014.03.20 23:59:59', 'BRENTSPOT', 'Brent 1', 1.29, 1.6, 0.8568833439015613, 91.09, 3.5, 2.053, -0.035],
 ['2014.04.01 23:59:59', '2014.04.02 23:59:59', 'BRENTSPOT', 'Brent 1', 1.39, 1.4, 0.8568833439015613, 89.59999999999998, 3.5, 2.053, -0.036000000000000004]]

If you want to use your file Brent 3.csv replace the call to io.StringIO with the path of the file.

answered Jul 24, 2018 at 9:05

tobiasraabe

4571 gold badge8 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

dignitas Over a year ago

By replacing csv with the directory of the file, now I get [[...\\Strategy Returns\\Brent 3.csv']] as a return. using pd.read_csv(io.StringIO(csv), header=None).values.tolist()

tobiasraabe Over a year ago

Do not only replace csv but io.StringIO(csv) with your path as a string. io.StringIO(csv) makes csv be treated like a .csv file on your disk.

dignitas · Accepted Answer · 2018-07-24 16:32:31Z

0

def create_list(stratname,directory):
    os.chdir(directory)
    temp = []
    fin = []

    for file in glob.glob("*.csv"): # search file
        if stratname in file:
            TDFile=open(file,"r")

            for line in TDFile: # go throug every line in file
                temp = []
                s = line.split(',')
                for x in s:
                    try:
                        temp.append(round(float(x),3))
                    except ValueError:
                        temp.append(x)
                fin.append(temp)
    return fin

This solved the problem. I added another array fin = [], which I appended the temporary value to for each line and emptying the values from temp = [] again for the next line in the file.

edited Jul 24, 2018 at 16:32

answered Jul 24, 2018 at 10:28

dignitas

651 silver badge7 bronze badges

Collectives™ on Stack Overflow

Append to a 2 dimensional array with mixed while reading a csv file in python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related