Questions tagged [protobuf]
Protocol Buffers are a serialization format with an interface description language developed by Google. Source: en.wikipedia.org/wiki/Protobuf
23 questions
1
vote
0
answers
546
views
Recommended generic approach to checking scalar field presence in protobuf 3
There are several questions related to presence field tracking of scalar fields in protobuf 3, but I didn't find any with generic default approach recommendation.
It may be useful to be able to track ...
- 133
0
votes
0
answers
183
views
Permissive vs Strict API
This question is similar to Permissive vs. Strict API Message validation. One difference is that we're using Protobuf.
A colleague and I are discussing standard message types for names.
One ...
- 171
2
votes
0
answers
575
views
Is non-contiguous protobuf field numbering an anti-pattern?
Is it an anti-pattern to have gaps in field numbers to aid readability?
The need being address by this idea is that metadata fields, which are "aspect" fields, can be placed in the proto ...
- 2,076
1
vote
1
answer
6k
views
Using protobuf to store large array of doubles
For my job, I'm working on a project that involves working with robot sensor data. We've been using protobuf to serialize and transfer data. I'm trying to set up a new protobuf type but this data ...
- 119
0
votes
0
answers
899
views
Why does Protobuf generate outer classes for Java?
I googled this and found questions like "How to use Protobuf message as java class without a java outer class?" (https://stackoverflow.com/questions/60312156/how-to-use-protobuf-message-as-...
- 101
-2
votes
2
answers
172
views
Prefer passing class or variable as const reference
In my company's code base there are examples of methods which receive some protobuf messages and use them in this fashion:
void doSomething(const A& a) {
// Only data from a.info() is used in ...
- 17
1
vote
0
answers
316
views
Separating generated flatbuffer/protobuf files from domain model - best practices
I'm using flatbuffers for the first time. I've generated my java classes and have tested out serializing / deserializaing a message. Now I'm thinking about how to integrate these in to my JavaFx and ...
- 137
0
votes
0
answers
103
views
Is it idiomatic to use protobufs as containers within a service?
I love gRPC, but I find every step of the protobuf process rather frustrating (particularly in Python). Even though they are structurally similar to data structures composed of lists and dicts, you ...
- 143
1
vote
0
answers
145
views
What's the most intuitive way to organize a set of endpoints to fetch users and their roles in groups?
What's the most intuitive way to organize a set of endpoints to fetch users and their roles in groups? i.e.
Users and their roles in a specified group.
Groups a user is in, and their role in each ...
- 471
2
votes
2
answers
783
views
What are the "scalar fields" and "composite fields" in JAVA?
Need to understand the below definition, from the protobuf tutorial:
mergeFrom(Message other): (builder only) merges the contents of other into this message, overwriting singular scalar fields, ...
- 123
1
vote
1
answer
1k
views
How to handle adding new field / partial data update for a gRPC API?
Assuming we have following message that will be used to update data and it just got updated to version 2.
message HelloRequest {
string name = 1; // version 1
bool is_valid = 2; // version 2
}
...
- 119
1
vote
1
answer
576
views
Why protobuf is said to be in binary format although we write it in text format similar to json
JSON
{
"hello": "world"
}
Protobuf
message HelloWorld {
required string name = 1;
}
Why do we say that protobuf uses binary format during network exchange and json don't, even though network always ...
- 29
2
votes
2
answers
759
views
Options for having model, parsers and serializers for a given data-format generated in multiple languages?
I am member of the Apache PLC4X (incubating) project. Here we are currently implementing multiple industry PLC protocols. While we initially focussed on creating Java versions of these, we are ...
- 137
2
votes
1
answer
171
views
Protobuf: compile specification once, use generated code everywhere?
Once written, a protobuf specification can be compiled using protoc to a variety of implementations in different languages (e.g. python, c++, go). That's great. But to compile that generated code, you ...
- 129
1
vote
0
answers
265
views
Can I add new function for protobuf generated struct
I am using protobuf definitions to define message types.
Structure looks like
message1 {
}
message2 {
bytes msg1 = 1
}
So message2 has a field which is a byte array representation of message1. I ...
- 111
0
votes
2
answers
2k
views
Consuming external APIs within the microservice architecture
I'm writing a small microservice-based application. One of the services is assigned the task of querying an external API and processing the JSON response (filtered list of houses). Since I'm using ...
- 125
3
votes
2
answers
940
views
Reverse engineering an ETL pipeline
I have a one-way large ETL pipeline in Scala where I start with protobuf schema A and end up with protobuf schema B. I want to create a many-to-many mapping between the fields of schema A and schema B,...
- 109
30
votes
1
answer
30k
views
Why Protobuf 3 made all fields on the messages optional?
The syntax 3 of protobuf made all the fields optional dropping the keywords required and optional from previous proto2 syntax. Reading some comments from developers it seems that it was done for ...
- 423
1
vote
2
answers
2k
views
Arbitrary protobuf message as byte-array over Websocket - how to determine actual message type upfront
Protobuf is nice and dandy, but it was not made with self-description in mind. Now this is totally okay if you're using a well-defined protocol and want to replace e.g. SOAP-messages or just want to ...
- 139
5
votes
1
answer
5k
views
Is it rational to convert protobuf into json to send it to a web server?
Protobuf is a good thing, it allows c++ devs to not bother with class serialization/deserialization, is fast and .proto format is very nice. It also allows automatic data validation. But the messages ...
- 221
0
votes
0
answers
167
views
Is it possible to use Protocol Buffers natively for message passing between a Go web service and an Android app?
I want to connect an Android app to a Go web service (I have a small social network in mind) and would like to pass data to and from the web service using Protocol Buffers, since it recently seems to ...
- 119
0
votes
1
answer
856
views
JSON or protocol buffer to replace the custom TCP message
Originally, We have two applications communication with TCP/IP, and both of them are implemented by C++. The messages between them are custom message type.
Now the client program will be changed to ...
- 139
24
votes
2
answers
17k
views
Protobuf design patterns
I am evaluating Google Protocol Buffers for a Java based service (but am expecting language agnostic patterns). I have two questions:
The first is a broad general question:
What patterns are we ...
- 1,128