Importing large Excel file to Python

Question

I am trying to import an Excel (.xlsx) file into the Spyder IDE. Everything works fine when I import small files, using openpyxl, but for this particular file (around 30MB and 800k rows) my system crashes.

Following is the part of code that imports:

from openpyxl import load_workbook 
wb = load_workbook(filename=path + 'cleaned_noTC_s_PERNO_Date.xlsx', data_only=True)

Can anyone please let me know what is wrong with this method and what else can I use to import the stated file?

any way you can share the excel file?

user308827
– user308827

2015-04-14 03:42:45 +00:00
Commented Apr 14, 2015 at 3:42 — user308827
– user308827, Commented Apr 14, 2015 at 3:42

Daniel Corin · Accepted Answer · 2017-07-06 06:13:06Z

1

Try using the excellent pandas library, it has very robust excel reading functionality and is pretty good with memory in my experience:

See here:

import pandas as pd
xl = pd.read_excel("file.xlsx")

edited Jul 6, 2017 at 6:13

Daniel Corin

2,0972 gold badges16 silver badges27 bronze badges

answered Apr 14, 2015 at 3:43

user308827

22.4k98 gold badges284 silver badges443 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Charlie Clark · Accepted Answer · 2015-04-14 03:40:24Z

0

It sounds like you're running out of memory. If you don't need to edit the file then you can use read_only mode, otherwise you'll need more memory.

answered Apr 14, 2015 at 3:40

Charlie Clark

19.7k4 gold badges56 silver badges64 bronze badges

Collectives™ on Stack Overflow

Importing large Excel file to Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related