How to manually create Git blob object and then read the contents using node.js like in the ProGit chapter on Git internals?

Question

I read the chapter called "Git Internals - Git Objects" in the ProGit book.

The final part, entitled "Object Storage", shows you how you can manually create a Git blob object, and then read the contents of that object. This is shown using Ruby.

I tried to do the same thing in node.

First I created a directory called my-git-tests, and in it I ran git init. I created one javascript file called s.js analogous to the commands in the chapter with Ruby, and here it is:

const crypto = require('crypto');
const path = require('path');
const fs = require('fs');
const zlib = require('zlib');

const content = 'what is up, doc?';
const header = `blob ${Buffer.from(content).length}\0`;
console.log('Header', header.length, header);

const store = header + content;

console.log('Store is ', store);

const hash = crypto.createHash('sha1');
const sha1 = hash.update(store, 'utf-8').digest('hex');

console.log('SHA-1 is ', sha1);

const objectPath = `.git/objects/${sha1.substr(0, 2)}/${sha1.substr(2)}`;

console.log('Path is ', objectPath);

fs.mkdirSync(path.dirname(objectPath));

let zlibCompress;

zlib.deflate(store, (err, buffer) => {
  if (!err) {
    zlibCompress = buffer.toString('base64');
    console.log('zlib: ', zlibCompress);

    fs.writeFile(objectPath, zlibCompress, function(err) {
      if (err) {
        console.log(err);
      }
      console.log('saved');
    });
  } else {
    console.log('Error compressing.');
  }
});

When I run this script, the output is

Header 8 blob 16
Store is  blob 16what is up, doc?
SHA-1 is  bd9dbf5aae1a3862dd1526723246b20206e5fc37
Path is  .git/objects/bd/9dbf5aae1a3862dd1526723246b20206e5fc37
zlib:  eJwFwYEBACAEBMCV8kKNQ8/+I3RXvKyxzJbU4yDF4AHF9sLC8rZ5Gh/tqwrk
saved

However, when I try to read the Git object: git cat-file -p bd9dbf5aae1a3862dd1526723246b20206e5fc37

I get

error: inflate: data stream error (incorrect header check)
error: unable to unpack bd9dbf5aae1a3862dd1526723246b20206e5fc37 header
fatal: Not a valid object name bd9dbf5aae1a3862dd1526723246b20206e5fc37

I'm not sure what I am doing wrong here.

The header, checsum, and path are correct, so the problem must be in generating the zlib-compressed data and writing it to the path. The use of base64 looks suspicious: the actual loose object files are not base64-encoded but rather just raw bytes. — torek
– torek, Commented Mar 1, 2020 at 5:42
This post, and the example code, while rather specific & technical, and obviously not popular with many people, was exactly what I was looking for as reference for some work I'm doing. Thank you! — mkopala
– mkopala, Commented Jan 9, 2021 at 19:12

Imaginane · Accepted Answer · 2020-03-07 14:17:22Z

3

Don't use base64.

Replace zlibCompress = buffer.toString("base64); with zlibCompress = buffer;

git cat-file will read this perfectly fine.

answered Mar 7, 2020 at 14:17

Imaginane

311 bronze badge

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to manually create Git blob object and then read the contents using node.js like in the ProGit chapter on Git internals?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related