I got this .log file. I don't know how to read them as DataFrame
id | create_date
-----+----------------------------
318 | 2017-05-05 07:03:27.556697
456 | 2017-07-03 01:50:07.966652
249 | 2017-05-03 13:57:32.567373
-
is it really in that format? with those ascii-art-ish lines to delimit rows and columns?filippo– filippo2018-06-19 05:50:22 +00:00Commented Jun 19, 2018 at 5:50
-
@filippo Yes, true story :3Long– Long2018-06-19 06:14:02 +00:00Commented Jun 19, 2018 at 6:14
Add a comment
|
2 Answers
pd.read_table("data.csv", sep="|", skiprows=[1], header=0, parse_dates=[1]).rename(columns=lambda x: x.strip())
id create_date
0 318 2017-05-05 07:03:27.556697
1 456 2017-07-03 01:50:07.966652
2 249 2017-05-03 13:57:32.567373
Parameters
sep="|"Use
|as column separatorskiprows=[1]Ignore the second row, which is just decorations and would be the most problematic to parse
header=0Read column names from the first row
parse_dates=[1]Convert
create_datecolumn into pandasdatetime64format (may be optional)rename(columns=lambda x: x.strip())Remove extra whitespaces from column names
You may want to add index_col=0 if you want to make id column your index instead of using a sequential one.
1 Comment
Long
Tks @filipo, It's not actually solve my problem but It gives me clues to solve it.
try this,
df=pd.read_csv('file_.csv',sep='|')
then you can remove -----+---------------------------- in many ways
df[df[' id ']!='-----+----------------------------']df[~df[' id '].str.startswith('-')]df.drop(0)# it won't work if your file contains-----+----------------------------in any other places for examplefooterdf[df[' create_date '].notnull()]# it won't work when your create_date column contains NaN by default.
Output:
id create_date
1 318 2017-05-05 07:03:27.556697
2 456 2017-07-03 01:50:07.966652
3 249 2017-05-03 13:57:32.567373
answered Jun 19, 2018 at 6:46
Mohamed Thasin ah
11.2k11 gold badges65 silver badges120 bronze badges