How to actually track S3 upload progress (JavaScript SDK)

Question

I'd like to be able to track an S3 file upload's progress (bytes uploaded out of bytes total).

Before anyone flags this as a dupe—it's not. Every other answer I've seen on StackOverflow is actually incorrect. If you do something like this:

s3
  .putObject(
    {
      Bucket: 'xyz',
      Key: 'wow.png',
      Body: data,
    },
    (err, data) => {
      console.log('done', err, data);
    }
  )
  .on('httpUploadProgress', progress => {
    console.log('Progress:', progress);
  });

You get a single progress update, showing the total size:

Progress { loaded: 1082019, total: 1082019 }

This is not useful. What I'm looking for is a more fine-grained report of upload progress like you would normally see during an upload (no matter the file size.. 1MB or 100MB):

0% out of 100%
3% out of 100%
7% out of 100%
9% out of 100%
(etc)

Please define "fine-grained report of upload progress". What information exactly do you want? — Raptor
– Raptor, Commented Feb 22, 2018 at 2:48
Edited. I'm looking for regular progress updates as the upload progresses. — ffxsam
– ffxsam, Commented Feb 22, 2018 at 2:53
Did you tried a ManagedUpload? docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3/… — Pablo
– Pablo, Commented Feb 22, 2018 at 3:08
I did, but it doesn't work for me because your part size has to be at least 5MB. And then it would only report on each 5MB part being done. That's not useful to a user. — ffxsam
– ffxsam, Commented Feb 22, 2018 at 3:09
BTW, adjusting part size isn't ideal either. If I dialed down the part size to 50KB to better track uploads, it would result in a ton of parts uploading at once which is probably not a good idea. Does anyone know how to access the raw XHR in S3? — ffxsam
– ffxsam, Commented Feb 22, 2018 at 16:46

ffxsam · Accepted Answer · 2018-02-23 02:11:50Z

3

Well, this is the answer (as lame as it is). Just sharing this as it could be a potential gotcha for other people.

This will not work in a Node.js environment! I've been testing this in that manner, and it just calls the httpUploadProgress callback once. Once I moved my code to an actual front-end client, it works as expected and prints the progress as the file uploads.

answered Feb 23, 2018 at 2:11

ffxsam

27.9k35 gold badges101 silver badges151 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

styks Over a year ago

github.com/aws/aws-sdk-js/issues/1945#issuecomment-367936328 - "In browsers, the SDK adds an event listener to the underlying XmlHttpRequest’s progress event. In Node, progress is detected by the body stream being read"

Rafael · Accepted Answer · 2019-06-12 12:29:07Z

2

Try this:

s3
.putObject({
        Bucket: 'xyz',
        Key: 'wow.png',
        Body: data,
    },
    (err, data) => {
        console.log('done', err, data);
    }
)
.on('httpUploadProgress', ({ loaded, total }) => {
    console.log(ContentType, 'Progress:', loaded, '/', total, `${Math.round(100 * loaded / total)}%`);
})

answered Jun 12, 2019 at 12:29

Rafael

1,32514 silver badges21 bronze badges

Collectives™ on Stack Overflow

How to actually track S3 upload progress (JavaScript SDK)

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related