2

Can some one tell me the best (fastest) way to send data from a pandas data frame to Oracle? With pandas to_sql, it's depressingly slow due to one insert statement per row.

I tried odo, but cannot get passed a compilation error (construct has no default compilation handler) and documentation is not helping. Here's a related post of mine with a question.

I know there is a sql loader option for Python, but have never run a sub process (does that work in the Jupyter notebook?).

Ultimately, I'm trying to automate loading data from a CSV into Oracle on a monthly basis.

Improve this question

1 Answer 1

Reset to default
2

You can use the cx_Oracle python module which can do an insert of a list using the executemany() method. So you can convert your dataframe to a list or just load your csv file as a list and then load it into Oracle.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

That would be great. I just tried that, actually, following this: oracle.com/technetwork/articles/dsl/python-091105.html Is the speed comparable to SQL Loader?
Yes, I would say the speed is comparable. I use it for a lot of different jobs and never have any issues.
Do I need a list of lists (from the data frame)?
You just need to make sure that whatever you pass into the executemany() method is of the python type list. Otherwise it won't work.
I have a column that is formatted as datetime64[ns]. How do I specify this as such in cursor.setinputsizes()? I'll ask a separate question if you prefer.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.