0

I have json file which will grow in future so i want to split it as parent and child in one folder and refer all child in parent file so i could manage multiple file rather than one big file. I am using this file for configurations

Following is example to demonstrate my requirement

Main.json

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
            },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
            }
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
} 

I want to split like below

Parent.json

{
    "store": {
        "book": [
            "$ref": "book1.json",
            "$ref": "book2.json"
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
}

book1.json

{
    "category": "reference",
    "author": "Nigel Rees",
    "title": "Sayings of the Century",
    "price": 8.95
}

I tried to display parent file but its giving error, I want that parent.json should display as main.json

# jq . parent.json
parse error: ':' not as part of an object at line 10, column 19
7
  • 1
    your Parent.json is invalid JSON - "book": [ ... ] contains records which supposed to be listed in object, i.e.: "book": { ... } Commented Nov 12, 2019 at 12:48
  • @Dmitry book is array so i am referring array elements, you can compare with Main.json Commented Nov 12, 2019 at 12:58
  • yes, it's an array, so it cannot contain objects records in the format "label": "value", only objects do. If you like to list your entries as they spelled (i.e. with labels), then it should be like this "book": [ {"$ref": "book1.json"}, {"$ref": "book2.json"} ] Commented Nov 12, 2019 at 13:01
  • Thanks for quick response, its not to only display json file i want to use parent.json as config file for other applications, So for those applications Main.json and Parent.json should be same. Splitting this will only help maintenance of this file. Commented Nov 12, 2019 at 13:01
  • 1
    hmm. But then there's a dilemma: if you like to keep your Parent.json not in a JSON compliant format (so it'll be partially compliant), then you cannot use json tools to parse it right (cause the file violates JSON semantic). Thus you have to rely on text based tools (like awk, sed) to perform required substitution. But then, you'll be exposed to risk of facing false positives/negatives sooner of later (cause those text aware tools do not understand/handle recurrent/nested formats). Commented Nov 12, 2019 at 13:20

2 Answers 2

2

Regarding @Dmitry's points (that [ "$ref": "book1.json" ] is not valid JSON), why don't you instead have

[
  { "$ref": "book1.json" },
  { "$ref": "book2.json" },
  ...
]

Beyond this, what you're asking for is a program that generates multiple files, one of which is Parent.json and the rest of which are bookn.json. Since jq can only provide output via one stream, you could generate one line of compressed JSON per output file, the first line being Parent.json, and the remaining ones each bookn.json. You'd then have to post-process each line and pipe it into the appropriate file, and you could uncompress the line at that point.

  1. Producing those multiple lines,
$ jq -c '.store.book as $books | .store.book = [
      { "$ref": "book\(range(1; $books | length + 1)).json" } ], $books[]' Main.json
{"store":{"book":[{"$ref":"book1.json"},{"$ref":"book2.json"}],"bicycle":{"color":"red","price":19.95}},"expensive":10}
{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}
{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99}
  1. Piping those lines into each their file, uncompressing each one,
$ jq -c '...' Main.json | (
      IFS= read -r parent
      jq . <<< "$parent" > Parent.json

      i=1
      while IFS= read -r book
      do
        jq . <<< "$book" > "book$i.json"
        i=$(($i+1))
      done
  )

My sh is a bit rusty, storing reads seems unnecessarily complicated.


Update: In the spirit of Thor's excellent solution where he splits the tasks of generating Parent.json and bookn.json into two tasks, and solves the second part with split which seems pretty cut for it, I thought I'd see if I could do something similar:

genbooks() {
  jq -c '.store.book[]' |
  perl -nlE '$i++; open my $fh, "|jq . >book$i.json"; say $fh $_; close $fh'
}
Sign up to request clarification or add additional context in comments.

Comments

1

You could do it all in one go with jq and coreutils, e.g. define two functions like this:

genparent() {
  jq '.store.book = [ { "$ref": "book\(range(1; .store.book | length + 1)).json" } ]' > Parent.json
}

genbooks() {
  jq -c '.store.book[]'           |
  split --numeric-suffix=1        \
        --suffix-length=1         \
        --additional-suffix=.json \
        --filter='jq . > $FILE'   \
        --lines=1 - book
}

And run it like this:

<Main.json tee >(genparent) >(genbooks) > /dev/null
ls

Output:

book1.json  book2.json  Main.json  Parent.json

Parent.json

{
  "store": {
    "book": [
      {
        "$ref": "book1.json"
      },
      {
        "$ref": "book2.json"
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  },
  "expensive": 10
}

book1.json

{
  "category": "reference",
  "author": "Nigel Rees",
  "title": "Sayings of the Century",
  "price": 8.95
}

book2.json

{
  "category": "fiction",
  "author": "Evelyn Waugh",
  "title": "Sword of Honour",
  "price": 12.99
}

1 Comment

This makes me thinks those two tasks should of course be split apart. :-)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.