1

i have a file "stats.json" with the following initial structure"

[{
"price": 66.55,
"created": "2018-02-24T14:32:57"
}]

With a bash/curl script i take every 10 minutes data from an api and save it to "temp.json"

{
"price": 88.55,
"created": "2018-02-24T15:32:57"
}

I would like to merge the temp.json (which is updated every 10min) and to fill the stats.json file. I tried to do it with "jq" but without success.

2
  • 1
    This is going to get slower and more expensive the longer your file is (and moreover, constant rewriting of old data increases risk of corruption or loss); I strongly suggest taking peak's advice, and changing to a format you can append to as a constant-time operation. Commented Feb 24, 2018 at 17:15
  • thanks for the heads up, the data older than 7-14 days will be purged. Commented Feb 24, 2018 at 18:10

1 Answer 1

4

One way to add the item in temp.obj to the array in stats.json:

jq —-slurpfile temp temp.obj '. + $temp' stats.json

Another way:

jq —s '.[0] + [.[1]]' stats.json temp.json

(If you want to overwrite stats.json, then I’d suggest using sponge as illustrated below.)

However, if at all possible, it would almost certainly be better to maintain stats.json as a log file in JSONL format: http://jsonlines.org/

That is, each object in temp.json would be appended as a single line to stats.json:

jq -c temp.json >> stats.json

This of course assumes that stats.json is initially empty or already in the JSONL format. To convert your existing stats.json to JSONL, you could do something like this:

cp -p stats.json stats.json.bak

jq -c .[] stats.json | sponge stats.json
Sign up to request clarification or add additional context in comments.

3 Comments

jq -c temp.json >> stats.json Is there a way to add an comma after the input?
Yes, of course, but why would you want to? That would break everything. m The idea is to use line breaks as the "delimiter".
@peak I guess I do understand @nelasx; I guess we both failed to understand (him and I) is that a file full of objects will also work as a file that has [{object},{object}] but your post made me aware of this possibility! tks

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.