1

I have a large csv file which repeats data every few lines, I want to split it into multiple data frame from where the headers are repeated: For example, I have these in CSV file:

Order   Time       Value
1       18:28:54    15
2       19:28:56    11
Order   Time       Value
3       18:17:13    42
4       19:17:14    50
5       20:40:12    30
Order   Time       Value
6       14:20:12    5

This sequence continues for more than 20 times in the csv file and I want to have separate data frame created automatically:

df1:
Order   Time       Value
1       18:28:54    15
2       19:28:56    11

df2:
Order   Time       Value
3       18:17:13    42
4       19:17:14    50
5       20:40:12    30

df3:
Order   Time       Value
6       14:20:12    5

And so on...

Thanks

1

1 Answer 1

1

we can split the DataFrame by the header lines.

  • first we filter out the header lines by df[df['Order']=='Order'],at this moment the result Dataframe keeps the index
  • we use the header index to split the orignal dataframe like a list,for example:the start index is 0,the next header line index is 2,so we split dataframe like this df[0:2]

but I have no idea about make new variable dynamic,so I append all splited dataframe to a list

df = pd.read_csv("data.csv")
#filter all line which is header
split_lines = df[df['Order']=='Order']

dfs = []
last_idx = 0
for idx,row in split_lines.iterrows():
    #split line with the index
    _df = df[last_idx:idx]
    last_idx = idx+1
    dfs.append(_df)

_df = df[last_idx:df.shape[0]]
dfs.append(_df)

the example df with index like this

    Order      Time     Value
0      1    18:28:54     15
1      2    19:28:56     11
2  Order      Time      Value
3      3    18:17:13     42
4      4    19:17:14     50
5      5    20:40:12     30
6  Order      Time      Value
7      6    14:20:12      5
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.