0

enter image description here

I have a particular data file that has 7 columns total: 1 column of time and 6 columns of data. As depicted in the image I attached, the first column represents time, while the next four columns represent the data. It seems that from the 4th column of data onwards, it gets wrapped to the bottom with a little black upwards arrow, followed by a repeat in time as the first column again and the remaining two data columns represents the 5th and 6th column of data.

I am unable to load the data in python using np.loadtxt. How can I load the data while correctly accessing and extracting the 5th and 6th column of the data? I'm suppose to generalize this for 400 columns of data eventually.

I tried data = np.loadtxt('temp_59324.dat',delimiter = None) and it returns an error. If I only ask for a total of 5 columns of data (1 column of time, 4 columns of actual data), then np.loadtxt loads correctly. But anything other than 5 results in an error

Edit: I attempted the approach by @NoName and it does not return anything. I suspect that the little black arrow isn't the right arrow used in the code. Copy pasting the black arrow from the notepad gives a red bullet point enter image description here

Edit 2: Here is the google drive link to the data file https://drive.google.com/file/d/1LqtSNxKkPIqT_lKlCxWmnLgfGZ_AGjY1/view?usp=drive_link

4
  • Anyway you must split the file in two parts by np.loadtxt each part with max_rows=xxx and skiprows=xxx respectively, and finally merge the arrays. Commented Aug 5, 2024 at 20:52
  • @StasSimonov Thank you for your response. Can you elaborate further on how I can do that? Commented Aug 5, 2024 at 20:56
  • Is the wrap line number a known constant value? Are the timestamps matched row by row in both parts of the file? Commented Aug 5, 2024 at 21:02
  • @StasSimonov Yes, it always wraps/ends at 1.9e-7 because that is the last time stamp in the simulation. The timestamps are matched row by row yes Commented Aug 5, 2024 at 21:04

2 Answers 2

0

For a special case with the known wrap_line number and the matching timestamps:

p1 = np.loadtxt(file_name, max_rows=wrap_line-1)
p2 = np.loadtxt(file_name, skiprows=wrap_line, usecols=[1,2])
p = np.concatenate((p1,p2),axis=1)
Sign up to request clarification or add additional context in comments.

Comments

0

I have created a sample file and tried the simulated the same situation,

Input File:

0.0 1.23 2.34 3.45 4.56
↑
0.0 5.67 6.78
0.1 2.34 3.45 4.56 5.67
↑
0.1 6.78 7.89
0.2 3.45 4.56 5.67 6.78
↑
0.2 7.89 8.90
0.3 4.56 5.67 6.78 7.89
↑
0.3 8.90 9.01

Python code:

import numpy as np


def load_wrapped_data(filename, num_columns):
    data = []
    current_row = []
    time = None

    with open(filename, 'r') as file:
        for line in file:
            values = line.strip().split()

            if len(values) == 1 and values[0] == '↑':
                continue

            if len(current_row) == 0:
                time = float(values[0])
                current_row.extend(map(float, values[1:]))
            else:
                current_row.extend(map(float, values[1:]))

            if len(current_row) == num_columns - 1:
                data.append([time] + current_row)
                current_row = []
                time = None

    return np.array(data)


filename = 'temp_59324.dat'
num_columns = 7

data = load_wrapped_data(filename, num_columns)

print(data)

This gives the below output,

[[0.   1.23 2.34 3.45 4.56 5.67 6.78]
 [0.1  2.34 3.45 4.56 5.67 6.78 7.89]
 [0.2  3.45 4.56 5.67 6.78 7.89 8.9 ]
 [0.3  4.56 5.67 6.78 7.89 8.9  9.01]]

8 Comments

Thank you for the response, this is what I needed, but for some reason this approach is not working for me. See my edited post
Is it possible to share a sample input file in text instead of image, in your question? because there are different types of up arrows in ascii, want to know the exact one in the file
I've included the data file in my original post on google drive. Thank you
pls give access to the file, i have requested in google
I just did. Thank you
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.