Merge two complex JSON objects with arrays

Question

I have the two following json as input:

{
  "one": {
    "vars": [
      {
        "name": "a",
        "value": "a"
      },
      {
        "name": "b",
        "value": "b"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "c",
        "value": "c"
      },
      {
        "name": "d",
        "value": "d"
      }
    ]
  },
  "extras": "whatever"
}

{
  "one": {
    "vars": [
      {
        "name": "e",
        "value": "e"
      },
      {
        "name": "f",
        "value": "f"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "g",
        "value": "g"
      },
      {
        "name": "h",
        "value": "h"
      }
    ]
  }
}

And I'd like to merge them in order to obtain the following result where each of the vars array of each section are merged together:

{
  "one": {
    "vars": [
      {
        "name": "a",
        "value": "a"
      },
      {
        "name": "b",
        "value": "b"
      },
      {
        "name": "e",
        "value": "e"
      },
      {
        "name": "f",
        "value": "f"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "c",
        "value": "c"
      },
      {
        "name": "d",
        "value": "d"
      },
      {
        "name": "g",
        "value": "g"
      },
      {
        "name": "h",
        "value": "h"
      }
    ]
  },
  "extras": "whatever"
}

Ideally but not mandatory:

the keys (here one and two) would be arbitrary and an undefined number of them could be present.
the vars array would not contain duplicate (based on name) and right precedence would be applied to override values from the first array.

I managed to merge the two objects and only 1 array with the following command but the key is hardcoded and I'm a bit stuck from there:

jq -s '.[0].one.vars=([.[].one.vars]|flatten)|.[0]' file1.json file2.json

peak · Accepted Answer · 2019-01-23 03:50:48Z

First, here is a solution which is oblivious to the top-level key names, but which does not attempt to avoid duplicates:

$A
| reduce keys_unsorted[] as $k (.;
    if .[$k] | (type == "object") and has("vars")
    then (.[$k]|.vars) += ($B[$k]|.vars) else . end )

Here of course $A and $B refer to the two objects. You can set $A and $B in several ways.

If you want to reorder the top-level keys, you can simply extend the above with a filter specifying the order, e.g.: {extras, two, one}.

To avoid duplicates, I'd suggest writing a helper function to do just that, as illustrated in the following section.

Avoiding duplicates

def extend(stream):
  reduce stream as $s (.;
    (map(.name) | index($s|.name)) as $i
    | if $i then .[$i] += $s
      else . + [$s]
      end) ;


$A
| reduce keys_unsorted[] as $k (.;
    if .[$k] | (type == "object") and has("vars")
    then (.[$k].vars) = ( .[$k].vars | extend(($B[$k].vars[])))
    else . end
  )

oguz ismail · Accepted Answer · 2019-01-22 12:45:30Z

2

jq -n 'input as $b | input
| .one.vars |= . + $b.one.vars
| .two.vars |= . + $b.two.vars' file2.json file1.json

file1.json must come after file2.json in order to preserve extras.

edited Jan 22, 2019 at 12:45

answered Jan 22, 2019 at 12:39

oguz ismail

51.8k16 gold badges64 silver badges83 bronze badges

2 Comments

aaaaahaaaaa Over a year ago

Any idea on how to make the keys and order arbitrary?

oguz ismail Over a year ago

@aaaaahaaaaa sorry, no idea.

Collectives™ on Stack Overflow

Merge two complex JSON objects with arrays

2 Answers 2

Avoiding duplicates

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Avoiding duplicates

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related