3

I have binary nodejs Buffer object that contains bitmap information. How do make image from the buffer and save it to file?

Edit:

I tried using the file system package as @herchu said but if I do this:

let robot = require("robotjs")
let fs = require('fs')

let size = 200
let img = robot.screen.capture(0, 0, size, size)


let path = 'myfile.png'
let buffer = img.image

fs.open(path, 'w', function (err, fd) {
  if (err) {
    // Something wrong creating the file
  }

  fs.write(fd, buffer, 0, buffer.length, null, function (err) {
    // Something wrong writing contents!
  })
})

I get

enter image description here

Improve this question

5 Answers 5

Reset to default
7

Although solutions by @herchu and @Jake work, they are extremely slow (10-15s in my experience).

Jimp supports converting Raw Pixel Buffer into PNG out-of-the-box and works a lot faster (sub-second).

const img = robot.screen.capture(0, 0, width, height).image;
new Jimp({data: img, width, height}, (err, image) => {
    image.write(fileName);
});
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I have this error : No matching constructor overloading was found. Please see the docs for how to call the Jimp constructor.
This is like 100000e10 times faster.
That didn't work well for me on Ubuntu 18.04. At first, the produced image was transparent. I've noticed the the alpha channel was always 0, so I updated it to 255 just to find that the colors are wrong. For some reason, the buffer created by robotjs uses bgra and not rgba. Using the following code fixed that for me and still worked way faster than other suggested solutions here: let fixedImg = new Uint8Array(img.length); for (let i=0; i < img.length; i+=4){ fixedImg[i] = img[i+2]; // r fixedImg[i+1] = img[i+1]; // g fixedImg[i+2] = img[i+0]; // b fixedImg[i+3] = 255; // a }
3

Note: I am editing my answer according to your last edits

If you are using Robotjs, check that its Bitmap object contains a Buffer to raw pixels data -- not a PNG or any other file format contents, just pixels next to each other (exactly 200 x 200 elements in your case).

I have not found any function to write contents in other format in the Robotjs library (not that I know it either), so in this answer I am using a different library, Jimp, for the image manipulation.

let robot = require("robotjs")
let fs = require('fs')
let Jimp = require('jimp')

let size = 200
let rimg = robot.screen.capture(0, 0, size, size)
let path = 'myfile.png'

// Create a new blank image, same size as Robotjs' one
let jimg = new Jimp(size, size);
for (var x=0; x<size; x++) {
        for (var y=0; y<size; y++) {
                // hex is a string, rrggbb format
                var hex = rimg.colorAt(x, y);
                // Jimp expects an Int, with RGBA data,
                // so add FF as 'full opaque' to RGB color
                var num = parseInt(hex+"ff", 16)
                // Set pixel manually
                jimg.setPixelColor(num, x, y);
        }
    }
jimg.write(path)

Note that the conversion is done by manually iterating through all pixels; this is slow in JS. Also there are some details on how each library handles their pixel format, so some manipulation was needed in the loop -- it should be clear from the embedded comments.

Improve this answer

5 Comments

I found out about this awesome library. I made a simple color bot then I decided to make something smarter so I set up template matching in nodejs . I can take a picture of my screen but the library returns a buffer. robotjs.io/docs/syntax#screencapturex-y-width-height. I am afraid I cannot use filesystem though. github.com/octalmage/robotjs/issues/222
I edited my question. I would really appreciate your help if you know something about C or other more difficult languages because I spent few hours on this problem already. :/
I have edited the question. The code is now based in yours, uses robotjs and produces a working PNG file. (It was not specified in the original question that your buffer was NOT a PNG file... but a raw robotjs buffer instead)
I accepted this answer because you explained a lot. Sadly it is way to slow when you get to pictures that are like 600x600 and above. I will use shelljs and generate the screenshots to file there. Then I will probably serve them with express server to my client. Or just emitting them using socket.io.
I added an alternate code sample below for processing the pixels that runs faster (should be fast enough for most practical uses). And thanks to @herchu for pointing me in the right direction to begin with. :)
3

Adding this as an addendum to accepted answer from @herchu, this code sample processes/converts the raw bytes much more quickly (< 1s for me for a full screen). Hope this is helpful to someone.

var jimg = new Jimp(width, height);
for (var x=0; x<width; x++) {
    for (var y=0; y<height; y++) {
        var index = (y * rimg.byteWidth) + (x * rimg.bytesPerPixel);
        var r = rimg.image[index];
        var g = rimg.image[index+1];
        var b = rimg.image[index+2];
        var num = (r*256) + (g*256*256) + (b*256*256*256) + 255;
        jimg.setPixelColor(num, x, y);
    }
}
Improve this answer

Comments

1

Four times faster! About 280ms and 550Kb for full screen 1920x1080, if use this script. I found this pattern when I compared 2 byte threads per byte to the forehead.

const robotjs = require('robotjs');
const Jimp = require('jimp');
const app = require('express').Router();

app.get('/screenCapture', (req, res)=>{
  let image = robotjs.screen.capture();
  for(let i=0; i < image.image.length; i++){
      if(i%4 == 0){
          [image.image[i], image.image[i+2]] = [image.image[i+2], image.image[i]];
      }
  }

  var jimg = new Jimp(image.width, image.height);
  jimg.bitmap.data = image.image;
  jimg.getBuffer(Jimp.MIME_PNG, (err, result)=>{
      res.set('Content-Type', Jimp.MIME_PNG);
      res.send(result);
  });
});

If you add this code before jimp.getBuffer you'll get about 210ms and 320Kb for full screen

  jimg.rgba(true);
  jimg.filterType(1); 
  jimg.deflateLevel(5);
  jimg.deflateStrategy(1);
Improve this answer

Comments

0

I suggest you to take a look on sharp as it has superior performance metrics over jimp.

The issue with robotjs screen capturing, which actually happened to be very efficient, is BGRA color model and not RGBA. So you would need to do additional color rotation.

Also, as we take screenshot from the desktop I can't imagine the case where we would need transperency. So, I suggest to ignore it.

const [left, top, width, height] = [0, 0, 100, 100]
const channels = 3
const {image, width: cWidth, height: cHeight, bytesPerPixel, byteWidth} = robot.screen.capture(left, right, width, height)
const uint8array = new Uint8Array(cWidth*cHeight*channels);
for(let h=0; h<cHeight; h+=1) {
  for(let w=0; w<cWidth; w+=1) {
    let offset = (h*cWidth + w)*channels
    let offset2 = byteWidth*h + w*bytesPerPixel
    uint8array[offset] = image.readUInt8(offset2 + 2)
    uint8array[offset + 1] = image.readUInt8(offset2 + 1)
    uint8array[offset + 2] = image.readUInt8(offset2 + 0)
  }
}
await sharp(Buffer.from(uint8array), {
  raw: {
    width: cWidth,
    height: cHeight,
    channels,
  }
}).toFile('capture.png')

I use intermediate array here, but you actually can just to swap in the result of the screen capture.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.