Upload image to s3 from url

Question

I am trying to upload a image, for which i have a url into s3

I want to do the same without downloading the image to local storage

filePath = imageURL;
          let params = {
            Bucket: 'bucketname',
            Body: fs.createReadStream(filePath),
            Key: "folder/" + id + "originalImage"
          };
          s3.upload(params, function (err, data) {
            if (err) console.log(err);
            if (data) console.log("original image success");
          });

expecting a success but getting error : myURL is a https publicly accesible url.

[Error: ENOENT: no such file or directory, open '<myURL HERE>']
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path:
   '<myURL HERE>' }

Could you please clarify what you are attempting to do? Is the source image coming from Amazon S3, or simply a web-accessible URL? Or is it simply coming from the local disk? It is strange that you are setting a filePath to a web URL. Feel free to edit your question to clarify details. — John Rotenstein
– John Rotenstein, Commented Sep 5, 2019 at 0:06
@JohnRotenstein i have to upload 2 images into MY s3 bucket. one link is a web accesible cloudinary image link. and another is a web accesible amazon s3 link. both are https. I intend to make the upload happen without storing the same in a local directory — Saloni Sikdar
– Saloni Sikdar, Commented Sep 5, 2019 at 6:15

John Rotenstein · Accepted Answer · 2019-09-05 22:57:14Z

1

There are two ways to place files into an Amazon S3 bucket:

Upload the contents via PutObject(), or
Copy an object that is already in Amazon S3

So, in the case of your "web accessible Cloudinary image", it will need to be retrieved, then uploaded. You could either fully download the image and then upload it, or you could do some fancy stuff with streaming bodies where the source image is read into a buffer and then written to S3. Either way, the file will need to be "read" from Cloudinary the "written" to S3.

As for the "web accessible Amazon S3 link", you could use the CopyObject() command to instruct S3 to directly copy the object from the source bucket to the destination bucket. (It also works within a bucket.)

answered Sep 5, 2019 at 22:57

John Rotenstein

273k28 gold badges456 silver badges541 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Evgeni Gomziakov · Accepted Answer · 2023-01-25 09:49:29Z

1

This will make a GET request to the image URL, and pipe the response directly to the S3 upload stream, without the need to save the image to the local file system.

const request = require('request');
const fs = require('fs');

const imageURL = 'https://example.com/image.jpg';
const s3 = new AWS.S3();

const params = {
    Bucket: 'bucketname',
    Key: "folder/" + id + "originalImage"
};

const readStream = request(imageURL);
const uploadStream = s3.upload(params).createReadStream();

readStream.pipe(uploadStream);

answered Jan 25, 2023 at 9:49

Evgeni Gomziakov

764 bronze badges

Comments

Chetan Buddh · Accepted Answer · 2023-01-25 15:07:03Z

You can use following snippet to upload files in S3 using the file URL.

import axios from 'axios';
import awsSDK from 'aws-sdk';

const uploadImageToS3 = () => {
  const url = 'www.abc.com/img.jpg';
  axios({
    method: 'get',
    url: url,
    responseType: 'arraybuffer',
  })
 .then(function (response) {
   console.log('res', response.data);
   const arrayBuffer = response.data;
   if (arrayBuffer) {
     awsSDK.config.update({
       accessKeyId: 'aws_access_key_id',
       secretAccessKey: 'aws_secret_access_key',
       region: 'aws_region',
     });

    const s3Bucket = new awsSDK.S3({
      apiVersion: '2006-03-01',
      params: {
        Bucket: 'aws_bucket_name'
      }
    });

    const data = {
      Body: arrayBuffer,
      Key: 'unique_key.fileExtension',
      ACL: 'public-read'
    };

    const upload = s3Bucket.upload(data, (err, res) => {
      if (err) {
        console.log('s3 err:', err)
      }
      if (res) {
        console.log('s3 res:', res)
      }
    })
  }
});
}

uploadImageToS3();

Collectives™ on Stack Overflow

Upload image to s3 from url

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related