merge multiple lists with jq into single list

Question

I have a aws cli bash loop generating multiple json lists, containing dictionaries. I can't get jq to merge the lists into one large list with all the dictionaries.

e.g.

[  { "key1": "value1", "key2": "value2" },
   { "key1": "value3", "key2": "value4" }
]
[  { "key1": "value5", "key2": "value6" }
]
[  { "key1": "value7" }
]

this gets piped to jq and i would like it to be merged to

[ { "key1": "value1", "key2": "value2" },
  { "key1": "value3", "key2": "value4" },
  { "key1": "value5", "key2": "value6" },
  { "key1": "value7" }
]

The snippet is not valid JSON. Properly quote the values in the input shown — Inian
– Inian, Commented Aug 13, 2020 at 10:42
I checked it there, the 2nd bit (result) is valid, the first is just 3 json lists, that is the issue. The bash loop runs aws cli and gets a new json list back everytime, i am trying to merge them into one using jq — Pieter
– Pieter, Commented Aug 13, 2020 at 11:22

J Quinn · Accepted Answer · 2021-06-02 22:37:47Z

6

Assuming you have jq 1.5, the solution posted here:

JQ How to merge multiple objects into one

provides the basic technique of using jq -n with the inputs operator.

cat FILE | jq -n '[inputs|.[]]'

does the trick. [inputs] glues the multiple results into a single list and the '.[]' strips the extra level of list that was added.

answered Jun 2, 2021 at 22:37

J Quinn

3494 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Pieter · Accepted Answer · 2021-09-22 10:30:58Z

-2

i hacked a fix using awk, there must me a way to do it with jq.

| awk 'NR==1{print}NR>1{sub(/^]/,"");sub(/^\[/,",");print}END{print "]"}' \

What it does,

skip first line "[" (leave it in place)
remove all "^]" at start of lines
replace "[" at start of line with "," new list starting
at the end close it all with "]"

edited Sep 22, 2021 at 10:30

answered Aug 13, 2020 at 11:20

Pieter

2,19219 silver badges20 bronze badges

Collectives™ on Stack Overflow

merge multiple lists with jq into single list

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related