Merge two complex JSON objects (using jq)

Question

I'm trying to replace objects in a complex JSON object. It seemed that the tool jq could be offering the perfect solution, but I'm really struggling with the right choice / chain of filters.

I have a complete configuration JSON object which looks like this (has some more keys in it, shortened it for illustration):

{
    "some-array": [
        {
            "name": "foo",
            "attr": "value"
        },
        {
            "name": "foo bar",
            "attr": "value"
        },
        {
            "name": "foo bar baz",
            "attr": "value"
        }
    ],
    "some-other-array": []
}

Now I have another object containing an array with updated objects which I need to merge with the full configuration in some way. I need to find the nested objects by name, add it if it does not exist yet and replace it if it does exist.

{
    "some-array": [
        {
            "name": "foo",
            "attr": "new-value",
            "new-attrib": "new-value"
        },
        {
            "name": "foo bar",
            "attr": "new-value"
        }
    ]
}

So, with the above example, my expected result would be:

{
    "some-array": [
        {
            "name": "foo",
            "attr": "new-value",
            "new-attrib": "new-value"
        },
        {
            "name": "foo bar",
            "attr": "new-value"
        },
        {
            "name": "foo bar baz",
            "attr": "value"
        }
    ],
    "some-other-array": []
}

I already tried select(."some-array"[].name == "foo") to begin with and a few other things as a jq filter, but I'm struggling to move forward here and would really appreciate some inspiration / an actual solution.

Can anyone tell me if what I'm trying to achieve is actually possible with jq or do I have to find another solution?

Community · Accepted Answer · 2020-06-20 09:12:55Z

Here is a solution to the updated problem. This solution assumes that the names are string-valued. It relies on two helper functions:

# array-to-hash
def a2h(f): reduce .[] as $x ({}; . + {($x | f): $x});

# hash-to-array
def h2a: . as $in | reduce keys_unsorted[] as $k ([]; . + [$in[$k]]);

The first of these creates a "hash" based on an input array, and the second implements the inverse operation.

With these helper functions, the solution can be written:

.["some-array"] |= (a2h(.name) + ($update|.["some-array"] | a2h(.name)) | h2a)

where $update is the "new" value. This solution relies on the "right-dominance" of object-addition.

Output

For the given example, the output is:

{
  "some-array": [
    {
      "name": "foo",
      "attr": "new-value",
      "new-attrib": "new-value"
    },
    {
      "name": "foo bar",
      "attr": "new-value"
    },
    {
      "name": "foo bar baz",
      "attr": "value"
    }
  ],
  "some-other-array": []
}

peak · Accepted Answer · 2017-05-22 09:18:53Z

1

Yes, it's possible, and in fact quite easy under various interpretations of the problem as originally stated.

The following solves the the problem as it was originally stated, with "it" being interpreted as .["some-array"] rather than its constituents.

Assuming $update holds the object with the updated information as shown, the update could be performed using this filter:

.["some-array"] = ($update | .["some-array"])

There are many ways to endow $update with the desired value.

edited May 22, 2017 at 9:18

answered May 19, 2017 at 14:22

peak

119k21 gold badges185 silver badges218 bronze badges

4 Comments

zwerch Over a year ago

Thanks a lot for your fast answer, but I can't get your solution to work. I tried cat example.json | jq --slurpfile update example-update.json '.["some-array"] = ($update | .["some-array"])' and I'm getting jq: error (at <stdin>:17): Cannot index array with string "some-array". Does it have to do something with --slurpfile?

zwerch Over a year ago

I tried --argfile instead of --slurpfile afterwards and that is getting really close to the solution, but instead of expanding "some-array" it actually replaced the content of the array, which is not what I want.

peak Over a year ago

@zwerch - You wrote "add it if it does not exist yet and replace it if it does exist". Maybe you should show the expected result. Regarding --slurpfile, please read the jq manual.

zwerch Over a year ago

Sorry for the misunderstanding. I appreciate your help and I added the expected result.

Collectives™ on Stack Overflow

Merge two complex JSON objects (using jq)

2 Answers 2

Output

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Output

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related