0

I'm trying to programmatically decompress a .zst file using typescript. This is the code I am using:

import { createReadStream, createWriteStream } from 'node:fs';
import { createZstdDecompress } from 'node:zlib';
import { pipeline } from 'node:stream/promises';
import { PathLike } from "fs";

export async function decompress_zstd(input: PathLike, output: PathLike) {
    const zstdDecompress = createZstdDecompress();
    const source = createReadStream(input);
    const destination = createWriteStream(output);
    await pipeline(source, zstdDecompress, destination);
}

decompress_zstd("./data/scratch/lichess_db_puzzle.csv.zst", "./data/scratch/lichess_db_puzzle.csv")
  .catch((err) => {
    console.error('An error occurred:', err);
    process.exitCode = 1;
  });

This is the file I am trying to decompress (Direct download: https://database.lichess.org/lichess_db_puzzle.csv.zst or information page: https://database.lichess.org/#puzzles)

When I run the decompression function I get the following error and 0byte empty output file:

An error occurred: Error: Unknown frame descriptor
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at ZstdDecompress.zlibOnError [as onerror] (node:zlib:190:17) {
  errno: 10,
  code: 'ZSTD_error_prefix_unknown'
}

When I try research why this may be most troubleshooting points to their being an issue with the file itself (Zstd decompression error - Unknown frame descriptor), however, I can decompress the file fine using the operating system which makes me think it isn't that.

Does anyone know how to solve this?

Thanks

1
  • If you have a valid zst file that Node's zst implementation fails on, but regular zlib doesn't, then that's a bug that you should head on over to github.com/nodejs/node/issues for. Commented Nov 4 at 16:48

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.