How do I import data from json format into R using jsonlite package

Question

I am doing this

newzips=fromJSON("http://media.mongodb.org/zips.json")

You can see the data yourself at http://media.mongodb.org/zips.json

and getting thus

str(newzips)
 List of 5
 $ city : chr "ACMAR"
 $ loc  : num [1:2] -86.5 33.6
 $ pop  : num 6055
 $ state: chr "AL"
 $ _id  : chr "35004\"}{\"city\":\"ADAMSVILLE\",\"loc\":[-86.959727,33.588437],\"pop\":10616,\"state\":\"AL\",\"_

Think you have to read it and convert line by line, or join it with commas and convert it... — Spacedman
– Spacedman, Commented Jul 1, 2014 at 16:00
Don't really understand the downvotes here - question is reproducible, output is given etc. Reasonable to expect a .json file is truly JSON... — Spacedman
– Spacedman, Commented Jul 1, 2014 at 16:15

Jeroen Ooms · Accepted Answer · 2015-03-28 20:42:17Z

7

This format is called jsonlines. You can import it using the stream_in function in jsonite:

library(jsonlite)
zips <- stream_in(url("http://media.mongodb.org/zips.json"))

If the server uses https, you can use the curl package:

library(jsonlite)
library(curl)
zips <- stream_in(curl("https://media.mongodb.org/zips.json"))

Datasets where each line is a record are usually nosql database dumps. Because they might be too large to parse all at once, they are meant to be imported line-by-line, which is exactly what jsonlite does.

edited Mar 28, 2015 at 20:42

answered Dec 7, 2014 at 0:47

Jeroen Ooms

33.3k36 gold badges146 silver badges220 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Rich Scriven Over a year ago

Absolutely. I just streamed it with ease. Took about 1.7 seconds

Spacedman · Accepted Answer · 2014-07-01 16:04:39Z

4

readLines + mush it into a JS array with some brackets and comma-separation:

> json = fromJSON(paste("[",paste(readLines("http://media.mongodb.org/zips.json"),collapse=","),"]"))
Warning message:
In readLines("http://media.mongodb.org/zips.json") :
  incomplete final line found on 'http://media.mongodb.org/zips.json'
> head(json)
        city                 loc   pop state   _id
1      ACMAR -86.51557, 33.58413  6055    AL 35004
2 ADAMSVILLE -86.95973, 33.58844 10616    AL 35005
3      ADGER -87.16746, 33.43428  3205    AL 35006
4   KEYSTONE -86.81286, 33.23687 14218    AL 35007
5   NEW SITE -85.95109, 32.94145 19942    AL 35010
6     ALPINE -86.20893, 33.33116  3062    AL 35014

answered Jul 1, 2014 at 16:04

Spacedman

94.7k12 gold badges148 silver badges232 bronze badges

Collectives™ on Stack Overflow

How do I import data from json format into R using jsonlite package

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related