0

I downloaded some information in json format, but it looks different from what I am regularly used to. The basic structures consists of two objects: an array of arrays without keys and an array of objects with key:value pairs, indicating the "keys" for the first array and their type.

{
  "datatable": {
    "data": [
      [
        "2022-04-26",
        118313,
        0,
        "QQQ",
        null,
        "BL6CD96",
        "ARCAVA4600V8",
        "XBUE",
        "INVESCO QQQ TRUST SE1-CEDEAR",
        "Invesco QQQ Trust Series 1",
        "False",
        "False"
      ],
      [
        "2022-04-26",
        56360,
        22669,
        "QQQ",
        "46090E103",
        "BDQYP67",
        "US46090E1038",
        "XNAS",
        "INVESCO QQQ TRUST SERIES 1",
        "Invesco QQQ Trust Series 1",
        "True",
        "False"
      ],
      [
        "2022-04-26",
        44307,
        25533,
        "IBM",
        "459200101",
        "2005973",
        "US4592001014",
        "XNYS",
        "INTL BUSINESS MACHINES CORP",
        "International Business Machines Corp",
        "True",
        "True"
      ]
    ],
    "columns": [{
        "name": "marketdate",
        "type": "Date"
      },
      {
        "name": "seckey",
        "type": "Integer"
      },
      {
        "name": "securityid",
        "type": "Integer"
      },
      {
        "name": "ticker",
        "type": "text"
      },
      {
        "name": "cusip",
        "type": "text"
      },
      {
        "name": "sedol",
        "type": "text"
      },
      {
        "name": "isin",
        "type": "text"
      },
      {
        "name": "mic",
        "type": "text"
      },
      {
        "name": "securityname",
        "type": "text"
      },
      {
        "name": "companyname",
        "type": "text"
      },
      {
        "name": "uslisted",
        "type": "text"
      },
      {
        "name": "innqgi",
        "type": "text"
      }
    ]
  },
  "meta": {
    "next_cursor_id": null
  }
}

Result I am trying to achieve:

{
  "datatable": {
    "data": [
      [
        "marketdate":"2022-04-26",
        "seckey":118313,
        "securityid":0,
        "ticker":"QQQ",
        "cusip":"null",
        "sedol":"BL6CD96",
        "isin":"ARCAVA4600V8",
        "mic":"XBUE",
        "securityname":"INVESCO QQQ TRUST SE1-CEDEAR",
        "companyname":"Invesco QQQ Trust Series 1",
        "uslisted":"False",
        "innqgi":"False"
      ],
...
  },
  "meta": {
    "next_cursor_id": null
  }
}

How can I convert this into a regular key=value JSON OR How do I parse this in Java so that I have a POJO where the variable names = "colName"?

Thanks a lot! Nikhil

4
  • 2
    Please, post an example (excerpt) of the result Commented Apr 27, 2022 at 17:31
  • Which library for parsing json are you using? Commented Apr 27, 2022 at 17:49
  • @Chaosfire I use Jackson 2.6.6 for parsing Commented Apr 27, 2022 at 18:49
  • @DimaParzhitsky added an example of the result I am also ok if I can directly parse it into a POJO Commented Apr 27, 2022 at 18:55

2 Answers 2

1

You need to map column names from second array to values from first array using indexes. First let's create POJO structure.

public class DataObject {

  private LocalDate marketDate;
  private int secKey;
  private int securityId;
  private String ticker;
  private String cusip;
  private String sedol;
  private String isin;
  private String mic;
  private String securityName;
  private String companyName;
  private String uslisted;
  private String innqgi;

  //getters and setters
}

Then:

public class DataWrapper {

  private List<DataObject> data;

  //getters setters
}

Response:

public class Response {

  private DataWrapper datatable;

  //getters setters

  //omitting meta
}

Then create deserializer to map column names to corresponding values:

public class ResponseDeserializer extends StdDeserializer<Response> {

  private static final DateTimeFormatter FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd");

  private final Map<String, BiConsumer<JsonNode, DataObject>> map = new HashMap<>();

  public ResponseDeserializer() {
    super(Response.class);
    this.initMap();
  }

  private void initMap() {
    map.put("marketdate", (jsonNode, dataObject) -> dataObject.setMarketDate(LocalDate.parse(jsonNode.asText(), FORMATTER)));
    map.put("seckey", (jsonNode, dataObject) -> dataObject.setSecKey(jsonNode.asInt()));
    map.put("cusip", (jsonNode, dataObject) -> dataObject.setCusip(jsonNode.asText()));
    //do the same for rest
  }

  @Override
  public Response deserialize(JsonParser parser, DeserializationContext context) throws IOException {
    JsonNode root = parser.getCodec().readTree(parser);
    ArrayNode dataArray = (ArrayNode) root.get("datatable").get("data");
    ArrayNode columnsArray = (ArrayNode) root.get("datatable").get("columns");
    List<DataObject> objects = new ArrayList<>(dataArray.size());
    for (int index = 0; index < dataArray.size(); index++) {
      ArrayNode data = (ArrayNode) dataArray.get(index);
      DataObject dataObject = new DataObject();
      for (int dadaIndex = 0; dadaIndex < data.size(); dadaIndex++) {
        JsonNode node = data.get(dadaIndex);
        String columnName = columnsArray.get(dadaIndex).get("name").asText();
        this.map.getOrDefault(columnName, (jsonNode, dataObject1) -> {}).accept(node, dataObject);
      }
      objects.add(dataObject);
    }
    DataWrapper wrapper = new DataWrapper();
    wrapper.setData(objects);
    Response response = new Response();
    response.setDatatable(wrapper);
    return response;
  }
}

Here i am using a Map to map column name to operation setting the value, but you could do it with reflection as well, for example.

A serializer to parse local date to the same format as in input:

public class LocalDateSerializer extends StdSerializer<LocalDate> {

  private static final DateTimeFormatter FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd");

  public LocalDateSerializer() {
    super(LocalDate.class);
  }

  @Override
  public void serialize(LocalDate value, JsonGenerator gen, SerializerProvider provider) throws IOException {
    gen.writeString(FORMATTER.format(value));
  }
}

Register serializers/deserializers and test result:

public class Main {

  public static void main(String[] args) throws Exception {
    ObjectMapper mapper = new ObjectMapper();
    SimpleModule module = new SimpleModule();
    module.addDeserializer(Response.class, new ResponseDeserializer());
    module.addSerializer(LocalDate.class, new LocalDateSerializer());
    mapper.registerModule(module);

    Response response = mapper.readValue(inputJson, Response.class);
    String json = mapper.writeValueAsString(response);
    System.out.println(json);
  }
}
Sign up to request clarification or add additional context in comments.

Comments

0
import org.json.JSONArray;
import org.json.JSONObject;
import java.util.ArrayList;
import java.util.HashMap;


class HashMapExample {

    private static HashMap<String, Integer> bigHashMap;

    public static void main(String[] args) {

        try {

            JSONObject jsonObject = getJsonFromSource();

            // Get datatable object from JSONObject
            JSONObject dataTable = (JSONObject) jsonObject.get("datatable");

            if (dataTable != null) {

                // Get JSONArray from JSONObject datatable
                JSONArray data = dataTable.getJSONArray("data");
                JSONArray columns = dataTable.getJSONArray("columns");

                mapToKeyValuePair(data, columns); // Map key to value
            }
        } catch (Exception exception) {
            exception.printStackTrace();
        }
    }

    private static void mapToKeyValuePair(final JSONArray dataArray, final JSONArray columnsArray) {

        // Check for equal lengths
        if ((dataArray != null) && (columnsArray != null)) {

            ArrayList<String> fieldNames = new ArrayList<>(); // ArrayList with field names
            ArrayList<String> dataValuesArrays = new ArrayList<>(); // ArrayList with the data values
            ArrayList<HashMap> wholeFinalArray = new ArrayList<>(); // The whole array with key pair value

            // Loop to get a JSONObject with all the column names
            for (int i = 0; i < columnsArray.length(); i++) {
                JSONObject jsonObjectColumn = (JSONObject) columnsArray.get(i); // Get JSONObject with column names
                fieldNames.add(jsonObjectColumn.get("name").toString()); // Get fieldNames from JSONObject above
            }

            for (int i = 0; i < dataArray.length(); i++) {
                JSONArray jsonArrayData = (JSONArray) dataArray.get(i); // Get JSONArray with data
                dataValuesArrays.add(jsonArrayData.toString()); // Add the data to an ArrayList
            }

            // Loop through the data values combined arrays
            for (String dataValuesArray : dataValuesArrays) {

                JSONArray singleDataArray = new JSONArray(dataValuesArray); // Get single data array

                for (int a = 0; a < singleDataArray.length(); a++) {
                    HashMap<String, String> item = new HashMap<>();
                    item.put(fieldNames.get(a), singleDataArray.get(a).toString());
                    wholeFinalArray.add(item);
                }
            }
        
            System.out.println(wholeFinalArray);
        }
    }

    private static JSONObject getJsonFromSource() {

        String jsonResponse = "{'datatable':{'data': [['2022-04-26', 118313, 0, 'QQQ', null, " +
            "'BL6CD96', " +
            "'ARCAVA4600V8', 'XBUE', 'INVESCO QQQ TRUST SE1-CEDEAR', 'Invesco QQQ Trust Series 1', 'False', 'False'],['2022-04-26', 56360, 22669, 'QQQ', '46090E103', 'BDQYP67', 'US46090E1038', 'XNAS', 'INVESCO QQQ TRUST SERIES 1', 'Invesco QQQ Trust Series 1', 'True', 'False'],['2022-04-26', 44307, 25533, 'IBM', '459200101', '2005973', 'US4592001014', 'XNYS', 'INTL BUSINESS MACHINES CORP', 'International Business Machines Corp', 'True', 'True']],'columns': [{'name':'marketdate', 'type':'Date'},{'name':'seckey', 'type':'Integer'},{'name':'securityid', 'type':'Integer'},{'name':'ticker', 'type':'text'},{'name':'cusip', 'type':'text'},{'name':'sedol', 'type':'text'},{'name':'isin', 'type':'text'},{'name':'mic', 'type':'text'},{'name':'securityname', 'type':'text'},{'name':'companyname', 'type':'text'},{'name':'uslisted', 'type':'text'},{'name':'innqgi', 'type':'text'}]}, 'meta':{'next_cursor_id':null}}";

        return new JSONObject(jsonResponse);
    }
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.