Caching JSON data

Question

I've never really cached data before, but I feel as though it will greatly improve my site.

Basically, I pull JSON data from an external API that helps the basic functionality of my website. This info doesn't really change that often yet my users pull the info from the API thousands of times a day. If it updated once a day, that would be fine. I want to run a cron job daily that will pull the info and update the cache.

I already tried a few different things, both pulled using PHP:

1) Store data in an SQL table
I had it working, but there's no reason why I should ping the database each time when I can just store it in basic HTML/JSON.

2) .JSON file (using fwrite)
I had it stored, but the only way this worked is if the .getJSON() callback function is prepended to the JSON data and then the data is surrounded by parentheses (making it jsonp, I believe).

Does anyone have any advice or any directions to lead me in? As I said, I've never really done anything like this so I don't even know if I'm remotely headed in the right direction.

Edit:

Okay so I talked to my hosting and since I'm on a shared hosting (dreamhost) I can't install memcached, which sucks. The only info they could give me was that if it is on http://pecl.php.net/ then I can most likely use it. They said APC is available. I'm not sure if this fits my problem. I'd like to be able to access the cache directly in jQuery. Thanks

Chris Ledet · Accepted Answer · 2011-07-22 19:03:07Z

6

You can use memcached. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. Very easy to implement and has a low system footprint.

answered Jul 22, 2011 at 19:03

Chris Ledet

11.6k7 gold badges43 silver badges47 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

switz Over a year ago

Nice, I'll look into it and report back with any issues. Thanks!

switz Over a year ago

I have 13 or so different years with about 50 data sets in each of them. Should I just throw it all together as one big JSON sorted by year, or separate data sets? Can I even do that in memcached?

Chris Ledet Over a year ago

You can make the key the year and JSOn data as the value. Which ever is less / cleaner code for you.

Justin ᚅᚔᚈᚄᚒᚔ · Accepted Answer · 2011-07-22 23:02:48Z

2

Since you can't use memcached, go back to your database option and store it in a table using the MEMORY engine.

answered Jul 22, 2011 at 23:02

Justin ᚅᚔᚈᚄᚒᚔ

15.4k8 gold badges55 silver badges64 bronze badges

Comments

Mike Ruhlin · Accepted Answer · 2011-07-22 19:08:06Z

1

try memcached. You can easily generate a string key and store whatever blob of json you want with it. works like a dictionary, except persists in memory.

There's an option to give it an expiration time (otherwise it just stays cached forever). So when the user requests the data, just check if it's stored in memcached. If it is, great, return it. If it's not, do whatever you do to build it, then put it in memcached with a 24 hour expiration.

answered Jul 22, 2011 at 19:08

Mike Ruhlin

3,5562 gold badges24 silver badges31 bronze badges

Comments

Jonathan M · Accepted Answer · 2011-07-22 19:07:08Z

0

If your data varies per user:

I've done this by storing an object in the $_SESSION array.

I attach a quick bit of logic that determines if the data expiry period is up. If so, it draws new data, serves and caches. If not, it draws from $_SESSION and serves it up.

edited Jul 22, 2011 at 19:07

answered Jul 22, 2011 at 19:02

Jonathan M

17.5k9 gold badges61 silver badges95 bronze badges

2 Comments

Jase Over a year ago

If you do this, you're storing the same data once per user rather than once for the entire site.

Jonathan M Over a year ago

Yep. My data was per user. Not sure what his is.

Nikolay Fominyh · Accepted Answer · 2011-07-22 19:33:31Z

0

Try Redis. And to store data easily without unexpected errors on set/get - encode it using base64.

answered Jul 22, 2011 at 19:33

Nikolay Fominyh

9,2969 gold badges72 silver badges104 bronze badges

1 Comment

Nikolay Fominyh Over a year ago

If it doesn't change really often - you can create CRON task, that pulls it from remote server once an hour for example. And store it in static JS file.

Hugo Mota · Accepted Answer · 2011-07-22 19:54:01Z

0

this to store:

file_put_contents($path, $json_text);

and this to restore:

$json_text = file_get_contents($path);
echo $json_text;

echo can be used to pass the json exactly as it comes from the http request. if you need to parse it into a variable (in javascript) you can use array = JSON.parse('<?php echo $json_text; ?>');. if you need to parse in php, use $array = json_decode($json_text);.

answered Jul 22, 2011 at 19:54

Hugo Mota

11.7k9 gold badges45 silver badges61 bronze badges

Collectives™ on Stack Overflow

Caching JSON data

6 Answers 6

3 Comments

Comments

Comments

2 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

3 Comments

Comments

Comments

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related