0

My mysql version is 8.0. I have an InnoDB table with more than 10 million rows of data, and the data file size is 2G.

I am using pd.to_sql to insert several thousand rows of data each time. When the index is enabled, the insertion speed is very slow.

How to temporarily disable the index? And after the insertion is complete, enable the index?

I tried the operation in How to disable index in innodb:

SET autocommit=0;
SET unique_checks=0;
SET foreign_key_checks=0;

But the index is still valid.

2
  • 3
    How to temporarily disable the index? Noway. You may drop the index and recreate it after insertion completed. PS. Primary key (which is clustered) must be stored anycase - if you drop it with another indices then complete table rewriting will occur during its re-creation. Commented Jun 10, 2021 at 4:36
  • It seems that only this method is feasible. Commented Jun 10, 2021 at 4:43

2 Answers 2

3

No, something else is going wrong.

Please provide more details of the code involved. Inserting a few thousand rows in a 10M-row table will not benefit from the high cost of rebuilding the index.

Please provide SHOW CREATE TABLE so we can see the indexes. UNIQUE indexes are handled differently. FOREIGN KEYs add overhead. Batching the rows helps a lot, but 10K rows is on the high-end of what is efficient. What is the PRIMARY KEY? Are the rows being inserted in any predictable order?

Sign up to request clarification or add additional context in comments.

1 Comment

I made a comparison, inserting a few thousand rows does not need to delete the index.
3
  1. Drop the index
  2. Insert your data
  3. Create your index

2 Comments

Not applicable to primary key or the index which is treated as primary.
I predict this to be slower than simply inserting the rows -- since it is only 0.1% of the total.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.