1

I am fetching JSON data from the web and I am confuse on which is the best and faster way to store and retrieve JSON. Since I am not I am not handling complex queries. Is it advisable to store as flat file data.json or it should be in the database?

BTW, I am using PHP.

Improve this question
5
  • It depends on where and when you need the data. If you need to retrieve it more than once, or if more than one person need to access it, then I would suggest a database. But if it's like a one time use or not a lot of people you could probably get away with a flat file. Commented Jan 9, 2015 at 16:09
  • Depends on what you want to DO. An equivalent to your question would be (in an email to the) "Doctor, I'm sick. What medicine should I take?" Yeah, well... the one that's appropriate for what you have... so, what DO you have (in mind)? Commented Jan 9, 2015 at 16:09
  • So you are getting it 'from the web' so is that via a web service? is it yours? I'd recommend a micro service for getting/storing json, to keep the frontend interface separate from the crud implementation on the backend Commented Jan 9, 2015 at 16:11
  • 1
    @slime, here's the scenario. Every 15 minutes I fetch JSON data from the web. Since My API to access the JSON data is very limited, I need to store it somewhere in my server to avoid over query. My application will be open for public so meaning, there might be concurrent users accessing the parsed JSON data in my website. Since I update it every 15 minutes, those stored JSON in my server will be replaced by new data I fetch . Commented Jan 9, 2015 at 16:14
  • 1
    @user3569641 you should use a database to store the data, in anyways it is a better option. Commented Jan 9, 2015 at 16:15

1 Answer 1

Reset to default
1

It depends on the size of the data and the frequency of access. Not sure about MySql, but Postgres of late has excellent json-processing capabilities, allowing you to query or even index directly into your json data. This is definitely the way to go if your documents are several kilobytes at most or have almost constant size to keep the storage engine happy.

For large objects it's better to stick to the filesystem, storing a brief of the document in the DB to allow fast operations on most important fields.

edit: since the data will be frequently replaced, it is best to avoid disk writes and use a fast though less-reliable RAM-only storage like Redis.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

By the way, It's more on document or text. Very small files. More on KB to 1MB per file
Well I need to research on Redis since I am a MysQL user and I have no idea on what Redis is.
@user3569641 It's definitely worth your time, these key-value stores are really useful on high loads.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.