Extracting only some keys from a JSON file in Bash

Question

So, I have a JSON file that looks like this ;

 {
    "data": { stuff that i need }
    "stuff not needed": {more stuff not needed}
    "data": {more stuff that i need}
 }

In short, the stuff that I need is inside the curly braces of the "data" key. How can I print this in a Linux shell command? Note, there are several "data" objects in my file, and I would like to extract data from all of them each one at a time.

The intended output would be like this

data {...}
data {...}

Like @Poshi said, use jq. This video has a great explaination about how to use jq : youtube.com/watch?v=EvpwhGeiH0U — Siddharth Dushantha
– Siddharth Dushantha, Commented Mar 13, 2019 at 20:27

nullPointer · Accepted Answer · 2019-03-14 08:10:02Z

1

as others suggested you should really use jq tool for parsing json format. However if you don't have access to the tool and/or can't install it, below's a very simple way treating the json as raw text (not recommended) and producing the output you want :

 grep "\"data\":" json_file | tr -d \"

answered Mar 14, 2019 at 8:10

nullPointer

4,5891 gold badge20 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

David C. Rankin · Accepted Answer · 2019-03-14 08:49:52Z

You can very simply use awk with the field-separator of "{" and the substr and length($2) - 1 to trim the closing "}".

For example with your data:

$ awk -F"{" '/^[ ]*"data"/{print substr($2, 1, length($2)-1)}' json
 stuff that i need
more stuff that i need

(note: you can trim the leading space before "stuff" in the 1st line if needed)

Quick Explanation

awk -F"{" invoke awk with a field-separator of '{',
/^[ ]*"data"/ locate only lines beginning with zero-or-more spaces followed by "data",
print substr($2, 1, length($2)-1) print the substring of the 2nd field from the first character to the length-1 character removing the closing '}'.

bash Solution

With bash you can loop over each line looking for a line beginning with "data" and then use a couple of simple parameter expansions to remove the unwanted parts of the line from each end. For instance:

$ while read -r line; do 
    [[ $line =~ ^\ *\"data\" ]] && { 
        line="${line#*\{}"
        line="${line%\}*}"
        echo $line 
    }
done <json

(With your data in the json filename, you can just copy/paste into a terminal)

Example Use/Output

 $ while read -r line; do
>     [[ $line =~ ^\ *\"data\" ]] && {
>         line="${line#*\{}"
>         line="${line%\}*}"
>         echo $line
>     }
> done <json
stuff that i need
more stuff that i need

(note: bash default word splitting even handles the leading whitespace for you)

While you can do it with awk and bash, any serious JSON manipulation should be done with the jq utility.

Walter A · Accepted Answer · 2019-03-14 12:00:34Z

1

With the given input, you can use

sed -rn 's/.*"(data)": (.*)/\1 \2/p' inputfile

answered Mar 14, 2019 at 12:00

Walter A

20.2k2 gold badges29 silver badges46 bronze badges

Collectives™ on Stack Overflow

Extracting only some keys from a JSON file in Bash

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related