70 questions
1
vote
0
answers
36
views
Compatibility between streamlit and protobuf
I am unable to use the print(tf.version.VERSION) to check the tensorflow version. Reason being tensorflow looks for runtime_version in protobuf (from what I have learnt) and that is only supported in ...
- 683
1
vote
0
answers
51
views
A generated protobuf python file is unable to locate another protobuf module it generated
I have a project set up like this with protos/ as a top level directory and the project that actually builds it under services/common-py.
├── protos
│ ├── common.proto
│ ├── doodad.proto
└── ...
- 8,429
0
votes
0
answers
89
views
Issue Decoding Protobuf Messages from Kafka in PyFlink Due to String Conversion
We are trying to consume Protobuf-encoded messages from a Kafka topic using PyFlink. However, we are encountering message parsing errors when attempting to decode the messages later in our pipeline.
...
2
votes
2
answers
2k
views
"TypeError: Couldn't build proto file into descriptor pool" with google-cloud-compute
I'm running a Python application on Ubuntu 22 with Python 3.13.0.
Since upgrading Python from 3.12, I get this error when running the application:
TypeError: Couldn't build proto file into descriptor ...
- 51
1
vote
1
answer
208
views
How is protobuf generating this method and is it impossible to get auto-complete for it?
I was looking at a code base (GCP SDK's monitoring API) trying to drill down to get familiar with some methods, but I hit a wall here: https://cloud.google.com/monitoring/custom-metrics/creating-...
- 32.3k
0
votes
1
answer
162
views
Incorrect Protobuf classes are getting generated
I am trying out my first Protobuf sample code from the Protobuf documentation for Python.
https://protobuf.dev/getting-started/pythontutorial/
This is my .proto file.
syntax = "proto2";
...
0
votes
0
answers
141
views
Protobuf message from CPP to python
I have define my protobuf message that is serialize in CPP and send to Python program thank to a wrapper. On CPP side, the protobuf message is generate as std::string
Here is how I generate in cpp my ...
1
vote
0
answers
590
views
protobuf installed but cannot be imported in Poetry environment
I created a Poetry environment with following toml file:
[tool.poetry]
...
[tool.poetry.dependencies]
python = "^3.10"
mlrun = "1.4.1"
pyyaml = {file = "PyYAML-5.4.1-cp39-...
- 3,613
2
votes
1
answer
916
views
Decode protobuf without proto files in python
I would like to decode protobuf.
Example of the protobuf data: 0a06282c0241057a10011805220d080510bea3f493062a03010c1628f1a6f493063002382b4001481482010f383634333233303532343736343839
I can Decoding ...
- 643
3
votes
1
answer
4k
views
Failing to Import Files Compiled from Protobuf in Python
My directory structure is as follows:
test
|-test.py
|-test.proto
|-test_pb2.py
|-__init__.py
|-comm
|-comm.proto
|-comm_pb2.py
|-__init__.py
both __init__.py is empty
and test....
- 357
0
votes
0
answers
1k
views
Python Protobuf `MessageToDict` And `MessageToJson` with `include_default_value_fields` Does Add Defaults To `google.protobuf.BoolValue` Fields
I am attempting to use MessageToJSON and MessageToDict with the including_default_value_fields argument to generate a Pandas Dataframe that then gets written to Parquet. I have a deeply nested data ...
- 8,598
1
vote
2
answers
352
views
Having an error when installing Scann on Google Colab
I installed scann on colab with code !pip uninstall protobuf>=3.20.3 but I had a error like that:
"ERROR: pip's dependency resolver does not currently take into account all the packages that ...
- 11
0
votes
1
answer
540
views
When constructing an instance of a Python protobuf message, I keep getting positional argument error
I'm using pb_2 and want to consturct instances of a message but keep facing positional argument error. I can construct instances from all other messages but when it comes to the following example, I ...
- 59
3
votes
0
answers
725
views
Different libraries require different version of Protobuf (dependency conflict)
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-api-core 2.11.0 ...
0
votes
0
answers
988
views
Match (lib)protoc version on Mac to Python protobuf==3.19.X
I'm using a Mac/Linux and I'm trying to get python code generated from a .proto file. When I run the protoc --version command in my terminal it returns libprotoc 22.5.
When I run a terminal command ...
- 2,138
0
votes
1
answer
879
views
Compiling Protobuf to Python giving unexpected results
Disclaimer: I am relatively new to Protocol Buffers, and python is not my language of choice. Most of my knowledge of protocol buffers has used them without compiling them, or having an IDE handle it ...
- 101
2
votes
2
answers
1k
views
Protobuf file not generating correct Python code with 'protoc --python_out' command
I have a .proto file defining a message called 'DeviationCreate'.
syntax = "proto3";
package proto;
option go_package = "proto";
option java_package = "deviationalert";
...
- 21
1
vote
1
answer
498
views
How to get type of a protobuf message from a raw protobuf buffer and the .proto files?
Let's say that I have dumped a protobuf message to a file called packet.bin .
Let's also say that I have the .proto files which describes many message types of which one is the type of the dumped ...
2
votes
1
answer
2k
views
How to collate several protobuf messages into one object
I have a a protobuf message in the form of:
message CustomMessage
{
SubMessageTypeA a_msg = 1;
SubMessageTypeB b_msg = 2;
SubMessageTypeC c_msg = 3;
}
I have several objects (assume 3)...
- 541
0
votes
1
answer
368
views
Cleaning-up sphinx-doc protobuf types
I feel like I had this figured out before but it's not working again, I am developing some code based on protobuf and grpc and in the documentation all of the types come out really messily:
...
- 9,464
1
vote
1
answer
239
views
How to convert or deserialize byte octets to string in python
I get a message from rocket mq, it's like the string show in the pic above or the below:
b'\x08\xd2\t\x12\x03Tim\x1a(\x08\x04\x12\x18Test ProtoBuf for Python\x1a\n31.10.2019'
The message from rocket ...
- 59
0
votes
1
answer
73
views
how to invalidate toxenv
I have some tox project that processes some protobuf in the install-deps phase and outputs some *pb.py codecs (custom script executed as install_command option in the config). When I'm updating my ...
- 4,055
0
votes
1
answer
324
views
SerializeAsString - server, ParseFromString - client. "Decode error" Exception occurred, but the data looks correct
Our application workflow uses protobuf for cross-language IPC.
We have a C++ application publishing data over Linux shared memory to various clients on the same host.
Data is published to shared ...
0
votes
1
answer
398
views
regex as protobuf message field name?
can we define regular expression in protobuf field name? I send the request as list of dictinary in client.py file
"cur_cur_bin" : [{"cur_cur_bin1_bin3_bin1" : 4,"...
1
vote
1
answer
390
views
Protoc.py compiles files differently depending on where it is called from
If protoc.py is called from powershell, the compiled file is missing about half of the code as opposed to when protoc.py is called from natively from python. I have a powershell script and a python ...
0
votes
1
answer
1k
views
google.protobuf validation - python
I am trying to implement a validation for protobuf files in Python.
I don't want to use an external package.
I tried:
desc_set = descriptor_pb2.FileDescriptorSet()
descriptor_bytes = str.encode(...
2
votes
1
answer
2k
views
Python Dict -> Protobuf Message Parse Error: Timestamp
I have a protobuf timestamp message that I'm trying to parse from a Python dictionary.
The target Protobuf message looks like this:
message LogStatusSnapshot {
google.protobuf.Timestamp time = 1;
...
- 33
5
votes
2
answers
2k
views
Python class generated by protoc cannot be imported in the code because of unresolved references (Mac OS)
I tried to use protocol buffers on my project and the problem I have is that when I use protoc to generate the python class. The file that's generated looks nothing like in the example provided by ...
- 403
3
votes
2
answers
3k
views
Latest Google protobuf not working on app engine
For my GAE app, I was updating the libs in my requirements.txt, and after doing so, I get error messages relating to protobuf.
TypeError: Descriptors cannot not be created directly. If this call
came ...
- 16.6k
1
vote
1
answer
1k
views
Is there a Protostuff-equivalent Protobuff library for Python?
In Java, libraries like protostuff allow you to generate buffers from a Java POJO approximately like so:
Schema<Foo> schema = RuntimeSchema.getSchema(Foo.class);
...
protostuff = ...
- 3,855
6
votes
1
answer
12k
views
How can I check if a protobuf message has a field defined
I'm working with a protobuf message that has some of the fields marked for deprecation with [deprecated = true].
To my understanding the field can still be used by some part of the code (maybe with a ...
- 541
0
votes
1
answer
934
views
Where to find [email protected]?
My python gcld3 package broke recently because it is linked against libprotobuf.30.dylib and a recent homebrew update replaced this lib with a newer version.
The package [email protected] has libprotobuf....
- 12.6k
6
votes
0
answers
684
views
How to parse a .proto file into a JSON/Python structure?
I have a .proto file, describing a given schema and following proto2 syntax.
I would like to read this file in python, and generate a structure that I can then manipulate (can be JSON, a python data ...
- 1,462
4
votes
1
answer
2k
views
How to use Value in the python implementation of protobuf
I have a proto file defined as:
syntax = "proto3";
import "google/protobuf/struct.proto";
package generic.name;
message Message {
uint32 increment = 1;
google.protobuf.Value ...
1
vote
1
answer
3k
views
Protobuf: ParseFromString method gives error "raise message_mod.DecodeError('Field number 0 is illegal.')" for message with oneof field
I have a kafka topic that has protobuf message of format:
message CreditTransaction {
string date = 1;
float amount = 2;
}
message DebitTransaction {
string date = 1;
float amount = 2;...
- 1,115
0
votes
1
answer
1k
views
I want to use "." in message field name of .proto file
I defined .proto file like:
syntax = "proto3";
message networkSliceArray
{
int32 DRB.UEThpDl.SNSSAI = 1;
int32 DRB.UEThpUl.SNSSAI = 2;
string networkSliceIdentifier = 3;
}
I want ...
0
votes
1
answer
362
views
PySpark-Streaming protobuf message gives error: AttributeError: 'bytearray' object has no attribute 'WhichOneof'
I am reading protobuf messages from kafka topic using py-spark streaming.
While using the msg.WhichOneOf gives error AttributeError: 'bytearray' object has no attribute 'WhichOneof'.
The schema ...
- 1,115
6
votes
1
answer
4k
views
Why python code generation from proto is not generating classes?
I'm currently trying to generate python code from a proto file.
My proto file looks like this:
syntax = "proto3";
package display;
message Hello {
uint32 version = 1;
uint32 value = 2;
...
- 63
0
votes
1
answer
115
views
ProtoBuf and TransLink
I've found this GTFS data that I want to use, but IDK how to set up protobuf (preferably in Python) to read the data. Any help would be greatly appreciated. Thanks.
https://gtfsrt.api.translink.com.au/...
- 430
3
votes
3
answers
2k
views
gRPC/protobuf generated Python files use incorrect import statements
I have a grpc repo included in a python project repo as a submodule at src/grpc/protobuf with the following example structure:
protobuf/
|
|-data_structs/
| |-example_structA.proto
| |-...
- 1,599
1
vote
0
answers
2k
views
What's the proper way to set a google.protobuf.Int32Value in python?
I'm using Google's Int32Value protobuf in python. I'm wondering what the proper way to set the value is. If we do it during initialization, then we can do
my_int_32_value = Int32Value(value=123)
If we ...
- 3,198
2
votes
1
answer
2k
views
How to get header from grpc responce with python
I am trying to get header of the gRPC response with the following code, and it doesn't work:
response = stub.GetAccounts(users_pb2.GetAccountsRequest(), metadata=metadata)
header = response.header()
...
0
votes
1
answer
326
views
How can I use protobuf to limit the values passed into it? (python)
I'm creating a python file to read in some json data. Then assigning that data to my protobuf messages. I want to pass in "start_hour": 8 and receive 8 and not EIGHT from my enum. Perhaps I'...
0
votes
1
answer
590
views
How to do client side load-balancing with custom ThreadPool in gRPC?
I have created a grpc python client with round robin load-balancing policy
self.channel = grpc.insecure_channel(self.host,options=[("grpc.lb_policy_name",
...
- 21
6
votes
1
answer
4k
views
Reverse engineering .proto files from pb2.py generated with protoc
Is it possible to get proto files from generated pb2.py with protoc? Will be the same reverse engineering possible for gRPC?
1
vote
2
answers
6k
views
Protobuf 3.19.1 and Python 3.10 issue
This is all about and issue when using the latest Python Protobuf (3.19.1) and Python 3.10, in Linux (tested in Fedora 35 and Ubuntu 20.04.
It broke our library but it can easily tested using the ...
- 599
0
votes
1
answer
2k
views
Get list of proto messages
I'm a newbie to protobufs and I can't seem to get it.
I have a proto file like this.
message Address{
string Home=1;
State state=2;
string Name=3;
enum State{
STATE_UNKNOWN=...
- 3
4
votes
2
answers
1k
views
Implicit relative imports inside of Protobuf-generated Python package make importing from outside impossible
I have a package that looks like this:
setup.py
requirements.txt
alphausblue/
api/
ripple/
org_pb2.py
org/
v1/
org_pb2_grpc.py
In org_pb2_grpc.py I ...
- 8,202
2
votes
1
answer
4k
views
Protocol Buffer ParseFromString function not reading complete binary file in Python
I am testing out Protocol buffers and trying to read a csv file, serialize it and write the output to a binary file and then read the binary file using ParseFromString. I am able to serialize and ...
- 913
0
votes
1
answer
1k
views
How to make a basic protobuf program work using Python as on Google's Developer's Website?
I am following the tutorial of protobuf using Python (there isn't one for JavaScript). It doesn't work... and I think it might be outdated as proto2 and as a Python 2 program. How to make it work?
So ...
- 152k