0

I'm just starting to learn MySQL and I have a script that loads a large CSV file (4 million rows) into a table and then adds a spatial index to the GeometryLatLon column. The spatial index creation takes around 4 hours to complete. However, when I perform the same operation on SQLite, the spatial index creation only takes around 10 minutes. Is this performance difference normal?

Here's the MySQL script I'm using:

DROP TABLE IF EXISTS uprns;
CREATE TABLE uprns (
    UPRN BIGINT NOT NULL,
    X_COORDINATE DOUBLE,
    Y_COORDINATE DOUBLE,
    LATITUDE DOUBLE,
    LONGITUDE DOUBLE,
    GeometryLatLon POINT NOT NULL
) ENGINE=InnoDB;

-- Load data from CSV
LOAD DATA INFILE '4_million_rows.csv'
INTO TABLE uprns
FIELDS TERMINATED BY ',' 
ENCLOSED BY '"' 
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(UPRN, X_COORDINATE, Y_COORDINATE, LATITUDE, LONGITUDE)
SET GeometryLatLon = ST_GeomFromText(CONCAT('POINT(', LONGITUDE, ' ', LATITUDE, ')'), 4326);

ALTER TABLE uprns MODIFY GeometryLatLon POINT NOT NULL SRID 4326;

-- ALTER TABLE uprns DROP INDEX GeometryLatLon;
ALTER TABLE uprns ADD SPATIAL INDEX (GeometryLatLon);

CREATE INDEX idx_uprns_UPRN ON uprns (UPRN);

Are there any potential performance optimizations I can make in my MySQL script to improve the speed of spatial index creation? Or are there any known differences in the way MySQL and SQLite handle spatial indexing that could explain this disparity in performance?

5
  • Are you even creating a spatial index in the MySQL case? The documented syntax is different dev.mysql.com/doc/refman/8.0/en/creating-spatial-indexes.html. Commented May 2, 2023 at 20:54
  • Yes, I’m using the second syntax in your link ALTER TABLE uprns ADD SPATIAL INDEX (GeometryLatLon);. Unless I’m missing something? Once this line is executed, spatial queries are much faster. Commented May 4, 2023 at 5:58
  • No problem then. 4 hours for a spatial index is a long time, especially because it should be possible to make SQLite to create index 20 times faster than now sqlite.org/forum/forumpost/af937a36b3. Commented May 4, 2023 at 6:05
  • Sorry, do you mean that it is a long time, but it is normal for MySQL? Commented May 4, 2023 at 7:50
  • No, I do not know what is normal for MySQL. MySQL is not normal for me. Commented May 4, 2023 at 7:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.