zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

Question

I'm trying through Windows terminal to decompress a large number of compressed files with zstd v1.4.0 and then 'ag' search over :

zstd -dc -r . | ag -z -i "term"

It gives me the following error while proceeding :

zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

I have spent hours looking for a solution, tried different options for the zstd command but can't solve this.

This may indicate that ag is quitting, can you normally check if ag -z "term" <file> is working for you? — Tarun Lalwani
– Tarun Lalwani, Commented Aug 3, 2019 at 6:26
It gives me errors : ERR: Cannot decompress zipped file or ERR: Zip files not yet supported — titibouboul
– titibouboul, Commented Aug 6, 2019 at 17:46
Yes, so zip and gzip is different and I believe ag supports gzip not zip — Tarun Lalwani
– Tarun Lalwani, Commented Aug 7, 2019 at 2:04
@titibouboul you should try to narrow the problem, is it working with 1 compressed file ? maybe using powershell is different (in the past cmd pipe was: write to a temporary file and reading it) ? — mpromonet
– mpromonet, Commented Aug 9, 2019 at 17:28

Alexis Wilke · Accepted Answer · 2024-11-11 22:20:54Z

3

+50

If you like to search over uncompressed files, you should use :

zstd -dc -r . | ag -i "term"

zstd will uncompress the supported files and ag will search over it (your command try to uncompress twice, once with zstd, once with ag)

You can get the supported format using zstd -vV that should give you something like :

*** zstd command line interface 64-bits v1.4.0, by Yann Collet ***
*** supports: zstd, zstd legacy v0.5+, gzip

Note: The zstd version I tried doesn't support zip.

You can get the supported compressed format by ag using ag -V, that should give something like :

ag version 2.1.0

Features:
  +jit +lzma +zlib

Note: The ag version I tried doesn't support zip.

edited Nov 11, 2024 at 22:20

Alexis Wilke

21.2k11 gold badges111 silver badges183 bronze badges

answered Aug 7, 2019 at 17:53

mpromonet

12.1k45 gold badges68 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

titibouboul Over a year ago

Thanks for this lengthy answer, it's really interesting. I've tried first solution, ie. removing -z from formula and I have same error.

mpromonet Over a year ago

@titibouboul: This is strange because in decompress.c the message Zip files not yet supported is printed in method decompressand this method is called only when -z is used search.c

titibouboul Over a year ago

I'm referring to the original error I'm trying to fix : zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

Collectives™ on Stack Overflow

zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related