Passing standard input to grpc_tools.protoc and --decode Option Usage

Question

The usage summary for the --decode option when --help is called with grpc_tools.protoc mentions that the option reads some binary from the standard input and writes it as text to the standard output. I'm currently trying to decode the contents of some .pb file and piping the standard output to some text file using grpc_tools.protoc.main(). While this could normally be done via protoc --decode=a.b.message_type file.proto < file.pb > file.txt, I wish to do this all within Python and am at a loss for how to pass the contents of the .pb file to grpc_tools.protoc.main().

I tried the following implementation but it doesn't work as grpc_tools.protoc interprets the .pb file argument as a .proto file:

grpc_tools.protoc.main(['protoc', '--decode=a.b.message_type', '--proto_path=./relative/path/to/proto', 'file.proto', 'path/to/file.pb'])

The above returns an error stating that protoc.py "Could not make proto path relative: /path/to/file.pb".

Including the path to the .pb with --proto_path=./relative/path/to/pb as an additional option in the above command allows grpc_tools.protoc.main() to run but it does not properly decode the contents of the .pb file as it had interpreted it as a .proto file and greets me with a warning: "No syntax specified for the proto file: file.pb.

I was expecting that there is some option or way to pass the contents of the .pb file to grpc_tools.protoc.main() but I cannot find any documentation about that.

is subprocess.communicate (to protoc directly, not to grpc_tools.protoc) an option? stackoverflow.com/a/8475367/23354 — Marc Gravell
– Marc Gravell, Commented May 22, 2024 at 7:17
I'd brought up the use of subprocess.Popen()/subprocess.communicate() but I was discouraged to use that for some reason (something about it may not always be an option to install protoc in the final environment) — lauragrass
– lauragrass, Commented May 30, 2024 at 4:55
pretty sure grpc_tools.protoc is not native python - it just calls into protoc, so... (here is the wrapper: github.com/grpc/grpc/blob/master/tools/distrib/python/…) — Marc Gravell
– Marc Gravell, Commented May 30, 2024 at 12:06

DazWilkin · Accepted Answer · 2024-05-22 18:16:05Z

0

Something like:

import subprocess

proto_path = "path/to/protoc"
proto_file = f"{proto_path}/file.proto"

service = "a.b"
message = "message_type"

command = [
    "python3",
    "-m",
    "grpc_tools.protoc",
    f"--proto_path={proto_path}",
    f"--decode={service}.{message}",
    proto_file,
]

with open("/path/to/file.pb", "r") as pb, open("/path/to/file.msg", "w") as m:
    subprocess.run(
        command,
        stdin=pb,
        stdout=m,
        check=True,
    )

To be clear this:

Requires a shell that
Runs a Python program that
Creates a shell that
Runs a Python program that
Runs protoc

There's considerable scope for errors and lack of error-handling.

It would be simpler to run protoc directly.

edited May 22, 2024 at 18:16

answered May 22, 2024 at 17:51

DazWilkin

41.7k8 gold badges61 silver badges113 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Passing standard input to grpc_tools.protoc and --decode Option Usage

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related