Laravel ajax file upload (excel file: 1M+ rows)

Question

I'm trying to create a proper progress bar for my excel file upload in Laravel I'm using ajax to upload the file and have gotten the progress bar for the upload part to "kind off" work. My only problem, is that when the file is uploaded (100% complete on the upload part) it still says "uploading...". It looks like uploadProgress is not jumping forward to success when it's done.

The file is processed in the back by laravel-excel, and it takes a very very long time to process big excel files with 1M+ rows.

Here is also a screenshot of it. I console logged the progress and it all went fine until it should go over to the success function in the ajax call.

What could I do to fix this? To limit the amount of rows in the excel file uploads are not something I can do as the website I'm creating has the need to upload files that has this many or more rows.

Also, my code for the ajax upload:

    // AJAX form upload
    var form = $('.form-import');
    var bar = $('.import-progress-bar');
    var status = $('.import-status');

    form.ajaxForm({
        beforeSend: function() {
            status.empty();
            bar.attr('value', 0);
        },
        uploadProgress: function(event, position, total, percentComplete) {
            bar.attr('value', percentComplete);
            status.html('Uploading...');
            console.log('Upload progress - '+ percentComplete + '%');
        },
        success: function() {
            status.html('Importing rows...');
        },
        complete: function() {
            status.html('Importing complete!');
        }
    });

Do you get any errors? Can you create a Fiddle? What is the jQuery version that you use? Are you sure status is queried correctly? — Lajos Arpad
– Lajos Arpad, Commented Feb 16, 2020 at 11:06
No errors. I can see that the request is not done because I can see in the database that rows are being added. But the file is uploaded, only the process is running. I'm using jquery 3.4.1 and also jquery-form. And yes, the status element is upating when I start the upload. — Kaizokupuffball
– Kaizokupuffball, Commented Feb 16, 2020 at 11:24
Yea, logging is on. There was no error. The ajax would just not go to the next function (success) after the upload progress reached 100%. But I'm fixing it using the solution posted by @Zoli below. — Kaizokupuffball
– Kaizokupuffball, Commented Feb 16, 2020 at 12:55

Zoli · Accepted Answer · 2020-02-17 10:31:49Z

2

I would use mysql LOAD DATA INTO, to process the excell (csv) file into database. I had a similar task, with several 5-600 MB csv files, which generated php runtime errors.

With the LOAD DATA INTO, I have imported these files in seconds, without issues.

8 Comments

Kaizokupuffball Over a year ago

This could work if the data I was importing didn't need validation I guess. But all the data in the cells are verified some way or another.

Zoli Over a year ago

You can import in a intermediate table, and process the data with a backend job. This way you can waludate the data too, and will be much lower hardware consuming process.

Kaizokupuffball Over a year ago

Ohh, okay. I think i understand. Thanks! I will try to do this and comment here if i get it working. :)

Zoli Over a year ago

I'd use a (generated) job id for the imported rows and would send back as a response to the UI(and save it in local storage). Then, from time to time its possible to query the intermediate table of how many rows are processed with that perticullar job_id.

Kaizokupuffball Over a year ago

Ahh, yea, that is a great idea to add a job_id to the uploaded file and then just query it later. Im not too familiar with jobs etc. in laravel, but im gonna take a look at it. Doesnt look like im getting the session stuff to work properly anyways, so thanks for the tip! :)

|

Collectives™ on Stack Overflow

Laravel ajax file upload (excel file: 1M+ rows)

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related