Flatten nested JSON array with JQ

Question

I have a JSON in the following format:

{
  "@version": "2.7.0",
  "site": {
    "@name": "http://api:9999",
    "@ssl": "false",
    "alerts": [
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "instances": [
          {
            "uri": "http://api:9999",
            "method": "POST",
            "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg"
          },
          {
            "uri": "http://api:9999",
            "method": "POST",
            "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ"
          }
        ],
        "count": "37"
      }
    ]
  }
}

I want to flatten the inner array - .site.alerts.instances to get the following JSON:

{
    "@name": "http://api:9999",
    "@ssl": "false",
    "alerts": [
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "uri": "http://api:9999",
        "method": "POST",
        "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg",
        "count": "37"
      },
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "uri": "http://api:9999",
        "method": "POST",
        "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ",
        "count": "37"
      }
    ]
  }

I was able to flatten the inner JSON array by using the following JQ pattern:

.site.alerts[] as $in | $in.instances[] as $h |  $in | del(.instances) as $in2 |  $h * $in2

This gave me a very close result:

{
  "uri": "http://api:9999",
  "method": "POST",
  "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg",
  "pluginid": "10094",
  "desc": "<p>Base64 encoded data was disclosed by the application/web server</p>",
  "count": "37"
}
{
  "uri": "http://api:9999",
  "method": "POST",
  "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ",
  "pluginid": "10094",
  "desc": "<p>Base64 encoded data was disclosed by the application/web server</p>",
  "count": "37"
}

But not a perfect result. The objects are not in an array, and fields from the parent object, which are not part of the array (e.g. .site.@name) are not included.

Can you help me improve the JQ pattern I created?

Thanks in advance!

Inian · Accepted Answer · 2018-07-23 16:51:23Z

9

That's a really good effort made. Your idea is right, you've made sure the .instances[] array is flattened, just use that logic to re-construct the JSON as you need as

jq '{ "@name" : .site."@name", 
      "@ssl"  : .site."@ssl", 
      "alerts": [.site.alerts[] as $in | $in.instances[] as $h | $in | del(.instances) as $in2 | $h * $in2 ]}' json

jqplay.org - URL

answered Jul 23, 2018 at 16:51

Inian

87k15 gold badges166 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Omer Levi Hevroni Over a year ago

Thank you so much! This is exactly what I was missing and couldn't figure out :)

Collectives™ on Stack Overflow

Flatten nested JSON array with JQ

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related