46

I'm new to using protobuf, and was wondering if there is a simple way to convert a json stream/string to a protobuf stream/string in Java?

For example,

protoString = convertToProto(jsonString)

I have a json string that I want to parse into a protobuf message. So, I want to first convert the json string to protobuf, and then call Message.parseFrom() on it.

4
  • You might want to take a look at FlatBuffers instead. Commented Jul 15, 2016 at 23:45
  • 1
    "is a simple way to convert a json stream/string to a protobuf stream/string? "Probably not. They work differently; protobuf fields aren't actually named in the serialized representation, but they are in a sequence. JSON is almost the opposite; fields are named, but the sequence generally doesn't matter. What you can do is take your generated protobuf classes and use Jackson mixins to annotate them and parse into them. Commented Jul 16, 2016 at 0:56
  • Related - if it helps, here is some utility to view protos as json. It has both, a Java API, as well as an installable GUI tool. You can also check the source code to see "how it does that": github.com/Zabuzard/ProtoToJson Commented Sep 22, 2022 at 18:34
  • Protobuf 3 has a built-in JSON mapping Commented Jan 2, 2023 at 13:00

5 Answers 5

56

With proto3 you can do this using JsonFormat. It parses directly from the JSON representation, so there is no need for separately calling MyMessage.parseFrom(...). Something like this should work:

JsonFormat.parser().merge(json_string, builder);
Sign up to request clarification or add additional context in comments.

3 Comments

When a value is invalid (say proto is boolean, but json value is integer), is there a way to configure the parser to ignore that field or set that field to null, instead of throwing com.google.protobuf.InvalidProtocolBufferException?
This doesnt seem to work for my class - I get com.google.protobuf.InvalidProtocolBufferException: JsonObject - any suggestions?
but it not supports for protobuf-javalite :(
17
//You can use this for converting your input json to a Struct / any other Protobuf Class    

import com.google.protobuf.Struct.Builder;
import com.google.protobuf.Struct;
import com.google.protobuf.util.JsonFormat;
import org.json.JSONObject;

JSONObject parameters = new JSONObject();

Builder structBuilder = Struct.newBuilder();
JsonFormat.parser().merge(parameters.toString(), structBuilder);

// Now use the structBuilder to pass below (I used it for Dialog Flow V2 Context Management)

1 Comment

While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value.
3

You can convert json string to Proto using builder and json String

Example :

YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser().merge(JsonString, protoBuilder);

If you want to ignore unknown json field then

 YourProto.Builder protoBuilder = YourProto.newBuilder();
 JsonFormat.parser()..ignoringUnknownFields().merge(JsonString, protoBuilder);

 

Another way is, to use mergeFrom method from ProtocolBuffer

Example :

YourProto.Builder protoBuilder = YourProto.newBuilder();
protoBuilder.mergeFrom(JsonString.getBytes());

Once it execute, you will get all the data in protoBuilder from json String

1 Comment

protoBuilder.mergeFrom does not seem to work with JSON
1

Since someone asked about getting the exception "com.google.protobuf.InvalidProtocolBufferException: JsonObject" when following Adam's advice--I ran into the same issue. Turns out it was due to the google protobuf timestamps. They are being serialized as an object containing two fields "seconds" and "nanos", since this isn't production code, I just got around this by parsing the JSON using jackson, going through the JSON object recursively and changing every timestamp from an object to a string formatted as per RFC 3339, I then serialized it back out and used the protobuf JSON parser as Adam has shown. This fixed the issue. This is some throwaway code I wrote (in my case all timestamp fields contain the word "timestamp", this could be more robust, but I don't care):

public Map<String, Object> fixJsonTimestamps(Map<String, Object> inMap) {
    Map<String, Object> outMap = new HashMap<>();
    for(String key : inMap.keySet()) {
        Object val = inMap.get(key);
        if(val instanceof Map) {
            Map<String, Object> valMap = (Map<String, Object>)val;
            if(key.toLowerCase().contains("timestamp") && 
                    valMap.containsKey("seconds") && valMap.containsKey("nanos")) {
                if(valMap.get("seconds") != null) {
                    ZonedDateTime d = ZonedDateTime.ofInstant(Instant.ofEpochSecond((int)valMap.get("seconds")).plusNanos((int)valMap.get("nanos")),
                            ZoneId.of("UTC"));
                    val = d.format(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"));
                }
                
            } else {
                val = fixJsonTimestamps(valMap);
            }
        } else if(val instanceof List && ((List) val).size() > 0 && 
                ((List) val).get(0) instanceof Map) {
            List<Map<String, Object>> outputList = new ArrayList<>();
            for(Map item : (List<Map>)val) {
                outputList.add(fixJsonTimestamps(item));
            }
            val = outputList;
        }
        outMap.put(key, val);
    }
    return outMap;
}

Not the most ideal solution but it works for what I am doing, I think I saw someone recommend using a different timestamp class.

1 Comment

Hey @Benjamin McCord. Welcome to Stack Overflow and thank you for your contribution. Have you tried improving Adam's answer with an edit instead of posting your own? It would surely help to make Adam's answer complete and more people would see your contribution as Adam's has the most upvotes.
1

Had to convert the following JSON to Protobuf. Used the following online tool to get the correct response Online Json to Protobuf Converter Tool
Input -

{
    "boolean": true,
    "color": "blue",
    "object": {
      "a": "b",
      "c": "d"
    },
    "string": "Hello World - JavaInUse"
}

Output -

fields {
  key: "boolean"
  value {
    bool_value: true
  }
}
fields {
  key: "color"
  value {
    string_value: "blue"
  }
}
fields {
  key: "object"
  value {
    struct_value {
      fields {
        key: "a"
        value {
          string_value: "b"
        }
      }
      fields {
        key: "c"
        value {
          string_value: "d"
        }
      }
    }
  }
}
fields {
  key: "string"
  value {
    string_value: "Hello World - JavaInUse"
  }
}

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.