I would like to convert an associative array in bash to a JSON hash/dict. I would prefer to use JQ to do this as it is already a dependency and I can rely on it to produce well formed json. Could someone demonstrate how to achieve this?
#!/bin/bash
declare -A dict=()
dict["foo"]=1
dict["bar"]=2
dict["baz"]=3
for i in "${!dict[@]}"
do
echo "key : $i"
echo "value: ${dict[$i]}"
done
echo 'desired output using jq: { "foo": 1, "bar": 2, "baz": 3 }'
There are many possibilities, but given that you already have written a bash for loop, you might like to begin with this variation of your script:
#!/bin/bash
# Requires bash with associative arrays
declare -A dict
dict["foo"]=1
dict["bar"]=2
dict["baz"]=3
for i in "${!dict[@]}"
do
echo "$i"
echo "${dict[$i]}"
done |
jq -n -R 'reduce inputs as $i ({}; . + { ($i): (input|(tonumber? // .)) })'
The result reflects the ordering of keys produced by the bash for loop:
{
"bar": 2,
"baz": 3,
"foo": 1
}
In general, the approach based on feeding jq the key-value pairs, with one key on a line followed by the corresponding value on the next line, has much to recommend it. A generic solution following this general scheme, but using NUL as the "line-end" character, is given below.
To make the above more generic, it would be better to present the keys and values as JSON entities. In the present case, we could write:
for i in "${!dict[@]}"
do
echo "\"$i\""
echo "${dict[$i]}"
done |
jq -n 'reduce inputs as $i ({}; . + { ($i): input })'
JSON keys must be JSON strings, so it may take some work to ensure that the desired mapping from bash keys to JSON keys is implemented. Similar remarks apply to the mapping from bash array values to JSON values. One way to handle arbitrary bash keys would be to let jq do the conversion:
printf "%s" "$i" | jq -Rs .
You could of course do the same thing with the bash array values, and let jq check whether the value can be converted to a number or to some other JSON type as desired (e.g. using fromjson? // .).
Here is a generic solution along the lines mentioned in the jq FAQ and advocated by @CharlesDuffy. It uses NUL as the delimiter when passing the bash keys and values to jq, and has the advantage of only requiring one call to jq. If desired, the filter fromjson? // . can be omitted or replaced by another one.
declare -A dict=( [$'foo\naha']=$'a\nb' [bar]=2 [baz]=$'{"x":0}' )
for key in "${!dict[@]}"; do
printf '%s\0%s\0' "$key" "${dict[$key]}"
done |
jq -Rs '
split("\u0000")
| . as $a
| reduce range(0; length/2) as $i
({}; . + {($a[2*$i]): ($a[2*$i + 1]|fromjson? // .)})'
Output:
{
"foo\naha": "a\nb",
"bar": 2,
"baz": {
"x": 0
}
}
jq, splitting behavior seems to have changed with respect to trailing delimiters: printf "a\0b\0c\0" | jq -Rs 'split("\u0000") produces ["a", "b", "c"] with version 1.6 but with 1.7 it produces ["a", "b", "c", ""]. For associative arrays that leads to {<usual stuff>, "": null}. I also tried with 1.5 and it behaves like 1.6. I composed the string passed to jq differently based on the version using range(0;lenght/2) for 1.6 and below, and range(0;(lenght-1)/2) for 1.7 and above.if (( ${jq_minor_version} < 7 )) ; then length_for_jq='(lenght-1)' ; else length_for_jq='(length)' ; fi. Where the jq_minor_version is from some bash string manipulation to get at the 7 in things like jq-1.7 or jq-1.7.1 which is jq_version=$(jq --version) ; jq_minor_patch=${jq_version##*.} ; jq_minor_version=${jq_minor_patch%%.*}. Finally change single quotes for double quotes around the code given to jq and replace length with ${length_for_jq}.You can initialize a variable to an empty object {} and add the key/values {($key):$value} for each iteration, re-injecting the result in the same variable :
#!/bin/bash
declare -A dict=()
dict["foo"]=1
dict["bar"]=2
dict["baz"]=3
data='{}'
for i in "${!dict[@]}"
do
data=$(jq -n --arg data "$data" \
--arg key "$i" \
--arg value "${dict[$i]}" \
'$data | fromjson + { ($key) : ($value | tonumber) }')
done
echo "$data"
This answer is from nico103 on freenode #jq:
#!/bin/bash
declare -A dict=()
dict["foo"]=1
dict["bar"]=2
dict["baz"]=3
assoc2json() {
declare -n v=$1
printf '%s\0' "${!v[@]}" "${v[@]}" |
jq -Rs 'split("\u0000") | . as $v | (length / 2) as $n | reduce range($n) as $idx ({}; .[$v[$idx]]=$v[$idx+$n])'
}
assoc2json dict
assoc2json() {, with no function keyword, avoids depending on ksh syntax supported by bash only for compatibility purposes.%q quotes strings for eval-style consumption by bash. I'm quite certain that there's a substantial number of inputs for which the processing done here is going to yield something that doesn't evaluate back to the original literal values. On the other hand, it'd be pretty straightforward to adopt the printf '%s\0' approach taken in my answer, combining it with the much shorter jq code here (just adopting the string-splitting bits)...
printf '%s\0'! Thanks for that idea! jq will turn the NULs into \u0000, so you'd have to split on that, which... you can! So the one-liner is now simpler: printf '%s\0' "${!arrayvar[@]}" "${arrayvar[@]}" | jq -Rs 'split("\u0000") | . as $v | (length / 2) as $n | reduce range($n) as $idx ({}; .[$v[$idx]]=$v[$idx+$n])'.bash 5.2 introduces the @k parameter transformation which, makes this much easier. Like:
$ declare -A dict=([foo]=1 [bar]=2 [baz]=3)
$ jq -n '[$ARGS.positional | _nwise(2) | {(.[0]): .[1]}] | add' --args "${dict[@]@k}"
{
"foo": "1",
"bar": "2",
"baz": "3"
}
This has been posted, and credited to nico103 on IRC, which is to say, me.
The thing that scares me, naturally, is that these associative array keys and values need quoting. Here's a start that requires some additional work to dequote keys and values:
function assoc2json {
typeset -n v=$1
printf '%q\n' "${!v[@]}" "${v[@]}" |
jq -Rcn '[inputs] |
. as $v |
(length / 2) as $n |
reduce range($n) as $idx ({}; .[$v[$idx]]=$v[$idx+$n])'
}
$ assoc2json a
{"foo\\ bar":"1","b":"bar\\ baz\\\"\\{\\}\\[\\]","c":"$'a\\nb'","d":"1"}
$
So now all that's needed is a jq function that removes the quotes, which come in several flavors:
I leave this last iterm as an exercise for the reader.
I should note that I'm using printf here as the iterator!
$ declare -A d=([$'foo\naha']=$'a\nb' [bar]=2 [baz]=$'{"x":0}')
$ a2j='$ARGS.positional | [.[:$n], .[$n:]] | transpose | map({ (first): last }) | add'
$ jq -nc "$a2j" --argjson n ${#d[@]} --args "${!d[@]}" "${d[@]}"
{"bar":"2","baz":"{\"x\":0}","foo\naha":"a\nb"}
This combine arbitrary bash keys by peak, use of jq arg by
bertrand martel, but use ${#d[@]}, ${!d[@]} and ${d[@]} to get
number of elements, keys and values in the array instead instead of
${d[@]@k} by oguz ismail.
This allow a simple jq expression without _nwise.
