reading and manipulating data from a text file in python

Question

I've made a function that reads a file and separates the first line from the rest. The two files which I'm using for this are aaa.txt and bbb.txt. I try to manipulate the data. I tried squaring each element of each string but I realize that I need to float it but I'm not sure why it won't just float. Where did I went wrong?

aaa.txt is below

test a line 1

3,6,8,99,-4,0.6,8

0,9,7,5,7,9,5

2,2,2,2,2,2,5

7,5,1,2,12,8,0.9

=====================

bbb.txt is as below

test b line 1

1,2,3,4,5,6,7,8

55,0,90,09,1,2,3,

8,9,7,6,8,7,6

3,43,5,8,2,4,1

======================

def mi_func(P):
    f=open(P, 'r')
    first = f.readline()
    restlines= f.readlines()
    f.close()
    return first, restlines


afirst,arest = mi_func('aaa.txt')
bfirst,brest = mi_func('bbb.txt')

print(arest)
print(brest)

##################
#convert a rest into a float here
#convert b rest into a float here
#################
for i in range(len(arest)):
    arest[i] = [float(x) for x in arest[i].strip().split(',')]

for i in range(len(brest)):
    brest[i] = [float(x) for x in brest[i].strip().split(',')]


p=[i**2 for i in arest]             #square each element of a rest
c=[i**2 for i in brest]             #square each element of b rest
print(p)
print(c)
['3,6,8,99,-4,0.6,8\n', '0,9,7,5,7,9,5\n', '2,2,2,2,2,2,5\n', '7,5,1,2,12,8,0.9\n']
['1,2,3,4,5,6,7,8\n', '55,0,90,09,1,2,3,\n', '8,9,7,6,8,7,6\n', '3,43,5,8,2,4,1']

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-36603657d444> in <module>()
     21 
     22 for i in range(len(brest)):
---> 23     brest[i] = [float(x) for x in brest[i].strip().split(',')]
     24 
     25 

<ipython-input-1-36603657d444> in <listcomp>(.0)
     21 
     22 for i in range(len(brest)):
---> 23     brest[i] = [float(x) for x in brest[i].strip().split(',')]
     24 
     25 

ValueError: could not convert string to float:

This is happening because in bbb.txt you've 55,0,90,09,1,2,3, notice the , in the end, your code tries to do float('') which is failing. — John Maged
– John Maged, Commented Mar 9, 2018 at 4:14
@JonasEphrem thanks a lot man, any help how to do that. its 4am and iive put a lot into this so far :( — tgmjack
– tgmjack, Commented Mar 9, 2018 at 4:20

DannyMoshe · Accepted Answer · 2018-03-09 05:07:04Z

I found your mistake, it seems that you have some new line characters and some empty characters and a '09' (in brest) that you are trying to cast to a float

So you need to remove them first. With your code it was just a couple simple tweaks:

def mi_func(P):
    f=open(P, 'r')
    first = f.readline()
    restlines= f.readlines()
    f.close()
    return first, restlines


afirst,arest = mi_func('aaa.txt')
bfirst,brest = mi_func('bbb.txt')

#This code will remove leading 0's and split numbers into a list 
#while eliminating any standalone new line characters:

arest = [x.lstrip('0').split(',') for x in arest if x != '\n']
brest = [x.lstrip('0').split(',') for x in brest if x != '\n']

for i in range(len(arest)):
    arest[i] = [float(x)**2 for x in arest[i] if x != '\n' and x!= '']

print(arest, 'a')

for i in range(len(brest)):
    brest[i] = [float(x)**2 for x in brest[i] if x != '\n' and x != '']

print(brest, 'b')
# This is your output
#[[81.0, 49.0, 25.0, 49.0, 81.0, 25.0], [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 25.0]] a
#[[3025.0, 0.0, 8100.0, 81.0, 1.0, 4.0, 9.0], [64.0, 81.0, 49.0, 36.0, 64.0, 49.0, 36.0]] b

Rahul · Accepted Answer · 2018-03-09 04:42:46Z

0

You can use this function to return firstline, float list from the text files you provided.

def mi_func(file_name):
    with open(file_name) as f:
        lines = f.readlines()
        float_list = [
            float(float_str)
            for line in lines[1:]
                for float_str in line.strip().split(',')
                    if float_str != None
        ]
        return lines[0].strip(), float_list
print(mi_func('aaa.txt'))
print(mi_func('bbb.txt'))

edited Mar 9, 2018 at 4:42

answered Mar 9, 2018 at 4:27

Rahul

11.7k5 gold badges63 silver badges100 bronze badges

1 Comment

Rahul Over a year ago

What do you mean not working. In you bbb.txt there is coma at end so I edited the code. Please try now.

Collectives™ on Stack Overflow

reading and manipulating data from a text file in python

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related