How to merge multiple Json files in folder in certain order

Question

I have several JSON files in a folder and I want to combine them in a single one following certain order given in order.json

I´ve tested the code below that merges all the files but in alphabetical order based on file name.

jq -s . *.json > Merged.json

This is the file Shoes.json

{ "document": { "product": "Shoes", "info": [ { "day": "1", "month": "1", "entry": "Some text about Shoes for day 1 and month 1", "code": "AJKD" }, { "day": "2", "month": "1", "entry": "Some text about Shoes for day 2 and month 1", "code": "KKGIR" } ] } }

This is the file Watches.json

{ "document": { "product": "Watches",   "info": [ { "day": "2", "month": "3",   "entry": "Some text about Watches for day 2 and month 3",   "code": "PEWQ" }    ] }     }

This is the file Accesories.json

{ "document": { "product": "Accesories",    "info": [ { "day": "7", "month": "2",   "entry": "Some text about Accesories for day 7 and month 2",    "code": "UYAAC" }   ] }     }

This is the file that gives the order I want to get in output order.json

{  
   "order":{  
      "product 1":"Watches",
      "product 2":"Accesories",
      "product 3":"Shoes"
   }
}

And the output file I´d like to get would be like this Merged.json:

{  
   "document":[  
      {  
         "product":"Watches",
         "info":[  
            {  
               "day":"2",
               "month":"3",
               "entry":"Some text about Watches for day 2 and month 3",
               "code":"PEWQ"
            }
         ]
      },
      {  
         "product":"Accesories",
         "info":[  
            {  
               "day":"7",
               "month":"2",
               "entry":"Some text about Accesories for day 7 and month 2",
               "code":"UYAAC"
            }
         ]
      },
      {  
         "product":"Shoes",
         "info":[  
            {  
               "day":"1",
               "month":"1",
               "entry":"Some text about Shoes for day 1 and month 1",
               "code":"AJKD"
            },
            {  
               "day":"2",
               "month":"1",
               "entry":"Some text about Shoes for day 2 and month 1",
               "code":"KKGIR"
            }
         ]
      }
   ]
}

Maybe someone could help me with this case.

Any help would be very appreciated.

peak · Accepted Answer · 2019-06-08 19:39:48Z

1

Here's a solution that uses order.json to determine which files are to be read by jq:

jq -n '{documents: [inputs.document]}' $(jq -r '.order[] + ".json"' order.json)

The approach exemplified immediately above has many advantages, but the above line makes various assumptions that might not be warranted. For example, it assumes there are no spaces in any of the file names.

Robust handling of file names

The following assumes bash-ful functionality:

mapfile -t args < <(jq -r '.order[] + ".json"' order.json)
jq -n '{documents: [inputs.document]}' "${args[@]}"

If your bash does not have mapfile, you could set the bash variable as follows:

args=()
while read -r f; do 
  args+=("$f")
done < <(jq -r '.order[] + ".json"' order.json)

edited Jun 8, 2019 at 19:39

answered Jun 8, 2019 at 1:32

peak

119k21 gold badges185 silver badges218 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ger Cas Over a year ago

I´m not the expert at all, but this second solution looks even better for me and shorter and easiest to understand :)

peak · Accepted Answer · 2019-06-09 04:17:13Z

1

The following solution has the advantage of not requiring any shell-specific functionality and only requiring a single invocation of jq. It assumes you want to be able to handle arbitrarily many files, as determined by order.json. Amongst other assumptions is that in the pwd, we can use the pattern [A-Z]*.json to select the relevant "documents".

jq -n --argfile order order.json '
  INDEX(inputs.document;  .product) as $dict
  | reduce $order.order[] as $product ([]; . + [$dict[$product]])
  | {document: .}
' [A-Z]*.json

def INDEX

If your jq does not have INDEX/2, then it might be a good time to upgrade; alternatively, you could simply add (prepend) its def:

def INDEX(stream; idx_expr):
  reduce stream as $row ({}; .[$row|idx_expr|tostring] = $row);

edited Jun 9, 2019 at 4:17

answered Jun 7, 2019 at 21:48

peak

119k21 gold badges185 silver badges218 bronze badges

3 Comments

Ger Cas Over a year ago

Hi again peak. Thanks for your help. I've tried with the same files I posted in question but I get this error

jq: error: INDEX/2 is not defined at <top-level>, line 2:   INDEX(inputs.document;  .product) as $dict jq: 1 compile error

. jq version it says [version 1.5-1-a5b5cbe] if it matters.

Ger Cas Over a year ago

Definitely I´ll need to upgrade. But meantime I tested your code adding the INDEX function and it works perfect. If you have a chance, since this solution is smaller, may you explain the logic please. What each step does? Thanks again.

peak Over a year ago

You can read up on INDEX and reduce in the standard jq documentation. You might find it helpful to use jqplay.org to consolidate your understanding. If you are unfamiliar with the concept of "reduction", you might find this blog post helpful -- blog.differentpla.net/blog/2019/01/11/jq-reduce -- but note that the first section contains an error. As an exercise, you might like to try to find it.

Collectives™ on Stack Overflow

How to merge multiple Json files in folder in certain order

2 Answers 2

Robust handling of file names

1 Comment

def INDEX

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Robust handling of file names

1 Comment

def INDEX

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related