Find and edit a Json file using bash

Question

I have multiple files in the following format with different categories like:

{
    "id": 1,
    "flags": ["a", "b", "c"],
    "name": "test",
    "category": "video",
    "notes": ""
}

Now I want to append all the files flags whose category is video with string d. So my final file should look like the file below:

{
    "id": 1,
    "flags": ["a", "b", "c", "d"],
    "name": "test",
    "category": "video",
    "notes": ""
}

Now using the following command I am able to find files of my interest, but now I want to work with editing part which I an unable to find as there are 100's of file to edit manually, e.g.

find . - name * | xargs grep "\"category\": \"video\"" | awk '{print $1}' | sed 's/://g'

Seems like it's caused by the capital V in Video? So it should be something like this "\"category\": \"video\"" instead. — swlim
– swlim, Commented Jul 8, 2016 at 23:18
@SWLim Sorry, That was typo, it's 'v'. The command I wrote works properly for getting me all files with category 'video'. Now I want a command to edit my json files flags. — Nabz C
– Nabz C, Commented Jul 8, 2016 at 23:21

sps · Accepted Answer · 2016-07-09 00:55:28Z

1

You can do this

find . -type f | xargs grep -l '"category": "video"' | xargs  sed -i -e '/flags/ s/]/, "d"]/'

This will find all the filnames which contain line with "category": "video", and then add the "d" flag.

Details:

find . -type f

=> Will get all the filenames in your directory

xargs grep -l '"category": "video"'

=> Will get those filenames which contain the line "category": "video"

xargs  sed -i -e '/flags/ s/]/, "d"]/'

=> Will add the "d" letter to the flags:line.

edited Jul 9, 2016 at 0:55

answered Jul 9, 2016 at 0:48

sps

2,7302 gold badges24 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Nabz C Over a year ago

Thats the answer I was looking for

Mike Robinson · Accepted Answer · 2016-07-09 02:45:32Z

0

"TWEET!!" ... (yellow flag thown to the ground) ... Time Out!

What you have, here, is "a JSON file." You also have, at your #!shebang command, your choice of(!) full-featured programming languages ... with intimate and thoroughly-knowledgeale support for JSON ... with which you can very-speedily write your command-file.

Even if it is "theoretically possible" to do this using "bash scripts," this is roughly equivalent to "putting a beautiful stone archway over the front-entrance to a supermarket." Therefore, "waste ye no time" in such an utterly-profitless pursuit. Write a script, using a language that "honest-to-goodness knows about(!) JSON," to decode the contents of the file, then manipulate it (as a data-structure), then re-encode it again.

answered Jul 9, 2016 at 2:45

Mike Robinson

9,0516 gold badges33 silver badges50 bronze badges

2 Comments

Nabz C Over a year ago

I did tried using programming languages but whenever I edit a JSON file I needs to be serialize, that was also a headache.

Mike Robinson Over a year ago

I don't understand what you mean by "needs to be serialize(d)." As far as I know, the only way to reliably "edit" a JSON file is to parse it, change the data-structure in memory, then rewrite the entire content of the file. Using a true programming language. There are many subtleties in the format when used with actual data, and clubbing it with "sed" or with some editor is extremely likely to break the file, causing it to be unreadable by anyone or anything.

kenorb · Accepted Answer · 2016-07-09 13:29:04Z

0

Here is a more appropriate approach using PHP in shell:

FILE=foo2.json php -r '$file = $_SERVER["FILE"]; $arr = json_decode(file_get_contents($file)); if ($arr->category == "video") { $arr->flags[] = "d"; file_put_contents($file,json_encode($arr)); }'

Which will load the file, decode into array, add "d" into flags property only when category is video, then write back to the file in JSON format.

To run this for every json file, you can use find command, e.g.

find . -name "*.json" -print0 | while IFS= read -r -d '' file; do
    FILE=$file
    # run above PHP command in here
done

answered Jul 9, 2016 at 13:29

kenorb

169k95 gold badges712 silver badges796 bronze badges

Comments

Community · Accepted Answer · 2017-04-13 12:51:57Z

0

If the files are in the same format, this command may help (version for a single file):

ex +':/category.*video/norm kkf]i, "d"' -scwq file1.json

or:

ex +':/flags/,/category/s/"c"/"c", "d"/' -scwq file1.json

which is basically using Ex editor (now part of Vim).

Explanation:

+ - executes Vim command (man ex)
:/pattern_or_range/cmd - find pattern, if successful execute another Vim commands (:h :/)
norm kkf]i - executes keystrokes in normal mode
- kk - move cursor up twice
- f] - find ]
- i, "d" - insert , "d"
-s - silent mode
-cwq - executes wq (write & quit)

For multiple files, use find and -execdir or extend above ex command to:

ex +'bufdo!:/category.*video/norm kkf]i, "d"' -scxa *.json

Where bufdo! executes command for every file, and -cxa saves every file. Add -V1 for extra verbose messages.

If flags line is not 2 lines above, then you may perform backward search instead. Or using similar approach to @sps by replacing ] with d.

See also: How to change previous line when the pattern is found? at Vim.SE.

edited Apr 13, 2017 at 12:51

CommunityBot

11 silver badge

answered Jul 9, 2016 at 2:07

kenorb

169k95 gold badges712 silver badges796 bronze badges

1 Comment

kenorb Over a year ago

@NabaChinde It works for your sample files, maybe you've it them in different format?

explorer · Accepted Answer · 2020-04-09 04:41:32Z

0

Using jq:

find . -type f | xargs cat | jq 'select(.category=="video") | .flags |= . + ["d"]'

Explanation:

jq 'select(.category=="video") | .flags |= . + ["d"]'
   # select(.category=="video") =>  filters by category field
   # .flags |= . + ["d"] => Updates the flags array

answered Apr 9, 2020 at 4:41

explorer

9528 silver badges21 bronze badges

Collectives™ on Stack Overflow

Find and edit a Json file using bash

5 Answers 5

1 Comment

2 Comments

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

2 Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related