I would like to stream the contents of an HTTP response to a variable. My goal is to get an image via request(), and store it in in MongoDB - but the image is always corrupted.
This is my code:
request('http://google.com/doodle.png', function (error, response, body) {
image = new Buffer(body, 'binary');
db.images.insert({ filename: 'google.png', imgData: image}, function (err) {
// handle errors etc.
});
})
What is the best way to use Buffer/streams in this case?
The request module buffers the response for you. In the callback, body is a string (or Buffer).
You only get a stream back from request if you don't provide a callback; request() returns a Stream.
See the docs for more detail and examples.
request assumes that the response is text, so it tries to convert the response body into a sring (regardless of the MIME type). This will corrupt binary data. If you want to get the raw bytes, specify a null encoding.
request({url:'http://google.com/doodle.png', encoding:null}, function (error, response, body) {
db.images.insert({ filename: 'google.png', imgData: body}, function (err) {
// handle errors etc.
});
});
var options = {
headers: {
'Content-Length': contentLength,
'Content-Type': 'application/octet-stream'
},
url: 'http://localhost:3000/lottery/lt',
body: formData,
encoding: null, // make response body to Buffer.
method: 'POST'
};
set encoding to null, return Buffer.
Have you tried piping this?:
request.get('http://google.com/doodle.png').pipe(request.put('{your mongo path}'))
(Though not familiar enough with Mongo to know if it supports direct inserts of binary data like this, I know CouchDB and Riak do.)
Nowadays, you can easily retreive a file in binary with Node 8, RequestJS and async await. I used the following:
const buffer = await request.get(pdf.url, { encoding: null });
The response was a Buffer containing the bytes of the pdf. Much cleaner than big option objects and old skool callbacks.
