4

I'm reading lines from a file to then work with them. Each line is composed solely by float numbers.

I have pretty much everything sorted up to convert the lines into arrays.

I basically do (pseudopython code)

 line=file.readlines()
 line=line.split(' ') # Or whatever separator
 array=np.array(line)
 #And then iterate over every value casting them as floats
      newarray[i]=array.float(array[i])

This works, buts seems a bit counterintuitive and antipythonic, I wanted to know if there is a better way to handle the inputs from a file to have at the end an array full of floats.

Improve this question
6
  • Please fix your pseudo-code. If you have a for statement, please actually write the for statement, rather than a comment. Are you asking for newarray=[ float(x) for x in array ] or newarray = map( float, array )? It's not clear what you're looking for. Commented Jun 2, 2011 at 10:53
  • @S.Lott The couple of answers I got implemented exactly what I asked for :). I did not write the for, because it could be a do, a for, or any other kind of loop instruction, I just wanted to point I was iterating over the variables. I basically want an array full of floats that are in a text file. Commented Jun 3, 2011 at 4:48
  • @Leon palafox: If I enter "executable pseudocode" into a search engine, the first hit I get is for Python. We don't need pseudopython. Commented Jun 3, 2011 at 5:14
  • @Johnsyweb And 10 years ago you probably got a Java Pseudo code, I rather be explicit in my terminology rather than hope python will keep being this popular ;) Commented Jun 3, 2011 at 5:21
  • @Leon palafox: Psedo-Python is made-up nonsense. It is not explicit. It's junk. "rather than hope python will keep being this popular" is exactly backwards. Millions of programmers know Python. No one knows the made-up pseudo-python except you. Please don't use made-up pseudo-junk when you could (with very little change) written simple, clear, well-known, obvious-to-many Python. Commented Jun 3, 2011 at 9:54

5 Answers 5

Reset to default
7

Quick answer:

arrays = []
for line in open(your_file): # no need to use readlines if you don't want to store them
    # use a list comprehension to build your array on the fly
    new_array = np.array((array.float(i) for i in line.split(' '))) 
    arrays.append(new_array)

If you process often this kind of data, the csv module will help.

import csv

arrays = []
# declare the format of you csv file and Python will turn line into
# lists for you 
parser = csv.reader(open(your_file), delimiter=' '))
for l in parser: 
    arrays.append(np.array((array.float(i) for i in l)))

If you feel wild, you can even make this completly declarative:

import csv

parser = csv.reader(open(your_file), delimiter=' '))
make_array = lambda row : np.array((array.float(i) for i in row)) 
arrays = [make_array(row) for row in parser]

And if you realy want you colleagues to hate you, you can make a one liner (NOT PYTHONIC AT ALL :-):

arrays = [np.array((array.float(i) for i in r)) for r in csv.reader(open(your_file), delimiter=' '))]

Stripping all the boiler plate and flexibility, you can end up with a clean and quite readable one liner. I wouldn't use it because I like the refatoring potential of using csv, but it can be good enought. It's a grey zone here, so I wouldn't say it's Pythonic, but it's definitly handy.

arrays = [np.array((array.float(i) for i in l.split())) for l in open(your_file))]
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

7

If you want a numpy array and each row in the text file has the same number of values:

a = numpy.loadtxt('data.txt')

Without numpy:

with open('data.txt') as f:
    arrays = list(csv.reader(f, delimiter=' ', quoting=csv.QUOTE_NONNUMERIC))

Or just:

with open('data.txt') as f:
    arrays = [map(float, line.split()) for line in f]
Improve this answer

2 Comments

numpy.loadtxt() is the right way to do it, if numpy can be used. +1
the last solution that includes map is so simple and works!
3

How about the following:

import numpy as np

arrays = []
for line in open('data.txt'):
  arrays.append(np.array([float(val) for val in line.rstrip('\n').split(' ') if val != '']))
Improve this answer

2 Comments

Make this even more useful by making it as a function,def f(type=float) and then use type(val) in the list comprehension.
Don't use the name type, it's reserved. def f(typ=float) or def f(tipe=float) would be better.
1

One possible one-liner:

a_list = [map(float, line.split(' ')) for line in a_file]

Note that I used map() here instead of a nested list comprehension to aid readability.

If you want a numpy array:

an_array = np.array([map(float, line.split(' ')) for line in a_file])
Improve this answer

2 Comments

you can omit ' ' argument for .split()
Sure you can, but I prefer making things explicit when answering SO questions. In my own code, which is largely read by people familiar with Python, or at least programming, I tend to leave it out.
0

I would use regular expressions

import re

all_lines = ''.join( file.readlines() )

new_array = np.array( re.findall('[\d.E+-]+', all_lines), float)

np.reshape( new_array, (m,n) )

First merging the files into one long string, and then extracting only the expressions corresponding to floats ( '[\d.E+-]' for scientific notation, but you can also use '[\d.]' for only float expressions).

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.