ModuleNotFoundError when Importing gRPC pb2 File Generated by Protoc in Python

Question

I'm encountering a ModuleNotFoundError when trying to import a generated gRPC module in Python.

In my project, I have a file structure like this:

from test.api.grpc_test_cases.specific_folder.api_pb2

specific_folder contains a proto_files folder with .proto files.
specific_folder also has test.py files and generated files together.

Here's what my protoc commands look like:

First, I generate D.proto because it imports into two other other proto files:

python3 \
-m grpc_tools.protoc \
-I proto_files \
--python_out=. \
--grpclib_python_out=. \
D.proto

Then, I generate the other protobuf files (A.proto, B.proto, C.proto), which import D.proto:

python3 \
-m grpc_tools.protoc \
-I proto_files \
--python_out=. \
--grpclib_python_out=. \
proto_files/A.proto \
proto_files/B.proto \
proto_files/C.proto

In my test file, when I try to import a the generated pb2 file, I get the following error:

ModuleNotFoundError: No module named 'api_pb2'

Here's how I'm trying to import it:

from file.innerfile.innermostfile.api.api_pb2 import ServiceRequest

I've tried several things to resolve this issue, including:

Checking the Python path to ensure the generated modules are in the search path.
Using both relative and absolute paths in the import statement.
Generating all protobuf files in a single compiler command.
Adding "from . import api_pb2" in the grpc.py file.

I expected that one of these approaches would resolve the ModuleNotFoundError issue, allowing me to import ServiceRequest to use in my pytest tests without any errors. I want to use ServiceRequest from the pb2 file to create a response object using pytest like this:

response = ServiceRequest()
response.name = "John"
response.id = 123
response.timestamp = "2023-09-18T12:00:00"

Please provide a minimal reproducible example. In my experience, Python, Protobufs and Packages are somewhat of a P-P-P-pain but most scenarios are sol(vu)ble. You should not need to stagger the protoc calls because of dependencies; one should suffice. You may need to add empty __init__.py files to some of the generated folders. protoc doesn't represent package's in Python-generated code which adds complexity. — DazWilkin
– DazWilkin, Commented Sep 19, 2023 at 0:59
Why --grpclib_python_out? This should be --grpc_python_out. — DazWilkin
– DazWilkin, Commented Sep 19, 2023 at 1:07

inkflaw · Accepted Answer · 2024-04-18 07:39:35Z

-1

Because protoc's path logic is depend on the location of the proto file.

The right way is let the proto file be in the same folder as the path to the python file you want to generate.

Example:

app
  ├── service (python module, directory)
  └── proto/
       ├── __init__.py       
       ├── api.proto
       ├── api_pb2.py
       └── api_pb2_grpc.py

Go to the app(Root directory, which you run python -m service command).

Run: protoc -I. --python_out=. --grpc_python_out=. ./proto/api.proto

All the . relative to the location of the proto file.

Also see: https://stackoverflow.com/a/76946302/15715806

edited Apr 18, 2024 at 7:39

answered Apr 18, 2024 at 7:36

inkflaw

655 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ModuleNotFoundError when Importing gRPC pb2 File Generated by Protoc in Python

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related