Read a json file and grep a string from matching pattern using bash

Question

I am trying to parse below big json file using bash

//part of json file
{"name": "Jenkins", "version": "Jenkins-2.22"},
{"name": "FitNesse", "version": "FitNesse-2.1"},
{"name": "Quint","version": "Quint-2.12"},
{"name": "Otto","version": "Otto-1.0"},
{"name": "Gradle","version": "Gradle-1.1"}

I am able read the line matching 'version' from json file

while read line; do
  if [[ $line =~ "version" ]] ; then
    echo $line; fi
done < $manifestfile

I want to store the values[Jenkins-2.22, FitNesse-2.1, Quint-2.12, Otto-1.0, Gradle-1.1] in array in my while loop

need help with tweaking my above code

is array mandatory? does the array need to processed after all? — RomanPerekhrest
– RomanPerekhrest, Commented May 5, 2017 at 18:54
yes @RomanPerekhrest all these tools version are displayed in different webpage (ex wiki.tools.com) once I have the version stored in an array then I can pass each array element and run seacrh against my wiki page — Mowgli
– Mowgli, Commented May 5, 2017 at 19:09

mklement0 · Accepted Answer · 2017-05-05 20:31:20Z

Assuming that the sample input above is inside a JSON array in file file.json, i.e.:

[
  {"name": "Jenkins", "version": "Jenkins-2.22"},
  {"name": "FitNesse", "version": "FitNesse-2.1"},
  {"name": "Quint","version": "Quint-2.12"},
  {"name": "Otto","version": "Otto-1.0"},
  {"name": "Gradle","version": "Gradle-1.1"}
]

^{Note: The only change required to make the commands below work in Bash 3.x too is to replace readarray -t versions with IFS=$'\n' read -d '' -ra versions}

Using Bash 4.x and `jq`:

jq must be installed first, but that's well worth the effort, if feasible: it enables sophisticated, robust JSON parsing.

# Extract the version numbers and read them into a Bash array.
readarray -t versions < <(jq -r '.[].version' file.json)

# Print the array.
printf '%s\n' "${versions[@]}"

The above yields:

Jenkins-2.22
FitNesse-2.1
Quint-2.12
Otto-1.0
Gradle-1.1

Using Bash 4.x and `awk`:

If you cannot install jq, the following awk solution is an alternative, but note that it is much less robust and relies on the JSON to be formatted exactly as above.
This warning applies to any solution using standard utilities (awk, grep, sed, ...) - only a dedicated JSON parser will work robustly.

readarray -t versions < <(awk -F\" 'NF>1 { print $(NF-1) }' file.json)

# Print the array.
printf '%s\n' "${versions[@]}"

mklement0 · Accepted Answer · 2017-05-05 19:35:19Z

2

sed -n 's/.*"\([^"]*\)"}.*/\1/p' <file>

edited May 5, 2017 at 19:35

mklement0

452k68 gold badges728 silver badges988 bronze badges

answered May 5, 2017 at 18:51

Michael

5,3452 gold badges16 silver badges37 bronze badges

Comments

RomanPerekhrest · Accepted Answer · 2017-05-05 20:53:53Z

jq is a proper tool to manipulate JSON data in Unix shell.

Let's say test.json file contains:

[
{"name": "Jenkins", "version": "Jenkins-2.22"},
{"name": "FitNesse", "version": "FitNesse-2.1"},
{"name": "Quint","version": "Quint-2.12"},
{"name": "Otto","version": "Otto-1.0"},
{"name": "Gradle","version": "Gradle-1.1"}
]

test_json.sh script (simplified version):

#!/bin/bash
versions=$(jq '.[] | .version' test.json)
versions_arr=($versions)  # contains an array of `version` key values

I must warn that the above approach ($versions) (creating an array upon string splitting) may fail if the string $versions has globbing characters in it.

The more "steady" approach would be using built-in read command with -a option to read the data wordwise into the specified array:

test_json.sh script ("steady" version):

#!/bin/bash
versions=$(jq '.[] | .version' test.json)
read -a versions_arr <<< $versions  # read `version` key values into array

Now, versions_arr is an array of version key values.
To print for e.g. the second array value you would do:

echo ${versions_arr[1]}

The output:

"FitNesse-2.1"

asayah · Accepted Answer · 2017-05-05 21:21:21Z

0

Use JQ for parsing your Json easily, just install JQ, then use :

if you have a json array, and in every element you have the field version, you can use :

 cat myJsonFile | jq '.[].version'

That will return a list of versions

If you want to process line by line use

while read line; do
  if [[ $line =~ "version" ]] ; then
    echo $line | jq .[] | jq ."version" ; fi
done < $manifestfile

answered May 5, 2017 at 21:21

asayah

1

Collectives™ on Stack Overflow

Read a json file and grep a string from matching pattern using bash

4 Answers 4

Using Bash 4.x and `jq`:

Using Bash 4.x and `awk`:

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Using Bash 4.x and jq:

Using Bash 4.x and awk:

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related

Using Bash 4.x and `jq`:

Using Bash 4.x and `awk`: