Parsing JSON from URL

Question

Is there any simplest way to parse JSON from a URL? I used Gson I can't find any helpful examples.

for example it will be like this gdata.youtube.com/feeds/api/videos/SIFL9qfmu5U?alt=json or from facebook graph — Peril
– Peril, Commented Sep 19, 2011 at 7:24

skynet · Accepted Answer · 2014-10-17 23:28:06Z

70

First you need to download the URL (as text):

private static String readUrl(String urlString) throws Exception {
    BufferedReader reader = null;
    try {
        URL url = new URL(urlString);
        reader = new BufferedReader(new InputStreamReader(url.openStream()));
        StringBuffer buffer = new StringBuffer();
        int read;
        char[] chars = new char[1024];
        while ((read = reader.read(chars)) != -1)
            buffer.append(chars, 0, read); 

        return buffer.toString();
    } finally {
        if (reader != null)
            reader.close();
    }
}

Then you need to parse it (and here you have some options).

GSON (full example):

static class Item {
    String title;
    String link;
    String description;
}

static class Page {
    String title;
    String link;
    String description;
    String language;
    List<Item> items;
}

public static void main(String[] args) throws Exception {

    String json = readUrl("http://www.javascriptkit.com/"
                          + "dhtmltutors/javascriptkit.json");

    Gson gson = new Gson();        
    Page page = gson.fromJson(json, Page.class);

    System.out.println(page.title);
    for (Item item : page.items)
        System.out.println("    " + item.title);
}

Outputs:

javascriptkit.com
    Document Text Resizer
    JavaScript Reference- Keyboard/ Mouse Buttons Events
    Dynamically loading an external JavaScript or CSS file

Try the java API from json.org:

try {
    JSONObject json = new JSONObject(readUrl("..."));

    String title = (String) json.get("title");
    ...

} catch (JSONException e) {
    e.printStackTrace();
}

edited Oct 17, 2014 at 23:28

skynet

9,9086 gold badges46 silver badges52 bronze badges

answered Sep 19, 2011 at 7:24

dacwe

43.6k12 gold badges118 silver badges142 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Peter Štibraný Over a year ago

'java' tag indicates that he doesn't want to use javascript.

dacwe Over a year ago

@Salah Yahya: Added GSON example + how to download an url

specializt Over a year ago

This will MANGLE the JSON and produce parsing-errors. Its completely unnecessary to build your own string from the received contents of a URL, GSON supports reading from BufferedReader directly. THIS EXAMPLE WILL NOT WORK IN EVERY CASE

Thufir Over a year ago

can you use generics to load an arbitrary JSON?

Hack-R Over a year ago

Needs to be updated. Link is broken and it's unclear what should replace the final "..."

ccpizza · Accepted Answer · 2023-04-19 14:04:57Z

14

GSON has a builder that takes a Reader object: fromJson(Reader json, Class<T> classOfT).

This means you can create a Reader from a URL and then pass it to Gson to consume the stream and do the deserialisation.

Only three lines of relevant code.

import java.io.InputStreamReader;
import java.net.URL;
import java.util.Map;

import com.google.gson.Gson;

public class GsonFetchNetworkJson {

    public static void main(String[] ignored) throws Exception {

        URL url = new URL("https://httpbin.org/get?color=red&shape=oval");
        InputStreamReader reader = new InputStreamReader(url.openStream());
        MyDto dto = new Gson().fromJson(reader, MyDto.class);

        // using the deserialized object
        System.out.println(dto.headers);
        System.out.println(dto.args);
        System.out.println(dto.origin);
        System.out.println(dto.url);
    }
    
    private class MyDto {
        Map<String, String> headers;
        Map<String, String> args;
        String origin;
        String url;
    }
}

^{If you happen to get a 403 error code with an endpoint which otherwise works fine (e.g. with curl or other clients) then a possible cause could be that the endpoint expects a User-Agent header and by default Java URLConnection is not setting it. An easy fix is to add at the top of the file e.g. System.setProperty("http.agent", "Netscape 1.0");.}

Maven dependency

<!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.10.1</version>
</dependency>

Gradle dependency

implementation 'com.google.code.gson:gson:2.10.1'

edited Apr 19, 2023 at 14:04

answered Jul 23, 2016 at 20:35

ccpizza

32.5k24 gold badges186 silver badges195 bronze badges

2 Comments

Sundar G Over a year ago

what if the json - args having the array elements ?

ccpizza Over a year ago

@Sundar: for arrays you'd use TypeToken: stackoverflow.com/a/10292824/191246

Stefan Schramm · Accepted Answer · 2014-01-21 20:08:43Z

10

You could use org.apache.commons.io.IOUtils for downloading and org.json.JSONTokener for parsing:

JSONObject jo = (JSONObject) new JSONTokener(IOUtils.toString(new URL("http://gdata.youtube.com/feeds/api/videos/SIFL9qfmu5U?alt=json"))).nextValue();
System.out.println(jo.getString("version"));

answered Jan 21, 2014 at 20:08

Stefan Schramm

1441 silver badge3 bronze badges

1 Comment

Selphiron Over a year ago

I think you have to add .openStream() after new URL(...). So it has to look like this:

JSONObject jo = (JSONObject) new JSONTokener(IOUtils.toString(new URL("http://gdata.youtube.com/feeds/api/videos/SIFL9qfmu5U?alt=json").openStream())).nextValue();

Confuse · Accepted Answer · 2014-09-21 04:30:47Z

Here is a easy method.

First parse the JSON from url -

public String readJSONFeed(String URL) {
    StringBuilder stringBuilder = new StringBuilder();
    HttpClient httpClient = new DefaultHttpClient();
    HttpGet httpGet = new HttpGet(URL);

    try {

        HttpResponse response = httpClient.execute(httpGet);
        StatusLine statusLine = response.getStatusLine();
        int statusCode = statusLine.getStatusCode();

        if (statusCode == 200) {

            HttpEntity entity = response.getEntity();
            InputStream inputStream = entity.getContent();
            BufferedReader reader = new BufferedReader(
                    new InputStreamReader(inputStream));
            String line;
            while ((line = reader.readLine()) != null) {
                stringBuilder.append(line);
            }

            inputStream.close();

        } else {
            Log.d("JSON", "Failed to download file");
        }
    } catch (Exception e) {
        Log.d("readJSONFeed", e.getLocalizedMessage());
    }
    return stringBuilder.toString();
}

Then place a task and then read the desired value from JSON -

private class ReadPlacesFeedTask extends AsyncTask<String, Void, String> {
    protected String doInBackground(String... urls) {

        return readJSONFeed(urls[0]);
    }

    protected void onPostExecute(String result) {

        JSONObject json;
        try {
            json = new JSONObject(result);

        ////CREATE A JSON OBJECT////

        JSONObject data = json.getJSONObject("JSON OBJECT NAME");

        ////GET A STRING////

        String title = data.getString("");

        //Similarly you can get other types of data
        //Replace String to the desired data type like int or boolean etc.

        } catch (JSONException e1) {
            e1.printStackTrace();

        }

        //GETTINGS DATA FROM JSON ARRAY//

        try {

            JSONObject jsonObject = new JSONObject(result);
            JSONArray postalCodesItems = new JSONArray(
                    jsonObject.getString("postalCodes"));

                JSONObject postalCodesItem = postalCodesItems
                        .getJSONObject(1);

        } catch (Exception e) {
            Log.d("ReadPlacesFeedTask", e.getLocalizedMessage());
        }
    }
}

You can then place a task like this -

new ReadPlacesFeedTask()
    .execute("JSON URL");

yazabara · Accepted Answer · 2015-10-15 08:12:56Z

1

public static TargetClassJson downloadPaletteJson(String url) throws IOException {
        if (StringUtils.isBlank(url)) {
            return null;
        }
        String genreJson = IOUtils.toString(new URL(url).openStream());
        return new Gson().fromJson(genreJson, TargetClassJson.class);
    }

answered Oct 15, 2015 at 8:12

yazabara

1,3535 gold badges21 silver badges42 bronze badges

Comments

Rami Sharaiyri · Accepted Answer · 2015-08-20 15:30:48Z

0

 import org.apache.commons.httpclient.util.URIUtil;
 import org.apache.commons.io.FileUtils;
 import groovy.json.JsonSlurper;
 import java.io.File;

    tmpDir = "/defineYourTmpDir"
    URL url = new URL("http://yourOwnURL.com/file.json");
    String path = tmpDir + "/tmpRemoteJson" + ".json";
    remoteJsonFile = new File(path);
    remoteJsonFile.deleteOnExit(); 
    FileUtils.copyURLToFile(url, remoteJsonFile);
    String fileTMPPath = remoteJsonFile.getPath();

    def inputTMPFile = new File(fileTMPPath);
    remoteParsedJson = new JsonSlurper().parseText(inputTMPFile.text);

edited Aug 20, 2015 at 15:30

answered Jul 14, 2015 at 7:42

Rami Sharaiyri

5465 silver badges16 bronze badges

Comments

Ecreators · Accepted Answer · 2017-09-06 21:03:58Z

0

I use java 1.8 with com.fasterxml.jackson.databind.ObjectMapper

ObjectMapper mapper = new ObjectMapper();
Integer      value = mapper.readValue(new URL("your url here"), Integer.class);

Integer.class can be also a complex type. Just for example used.

answered Sep 6, 2017 at 21:03

Ecreators

1

Comments

canadiancreed · Accepted Answer · 2017-02-07 08:11:55Z

-7

A simple alternative solution:

Paste the URL into a json to csv converter
Open the CSV file in either Excel or Open Office
Use the spreadsheet tools to parse the data

edited Feb 7, 2017 at 8:11

canadiancreed

1,9847 gold badges43 silver badges64 bronze badges

answered Mar 26, 2015 at 10:38

gareth

1

2 Comments

Piotr Dajlido Over a year ago

This is a very low quality answer. I don't think that using Excel is a complex solution to this problem.

zundi Over a year ago

promoting a json to csv converter maybe?

Collectives™ on Stack Overflow

Parsing JSON from URL

8 Answers 8

5 Comments

Maven dependency

Gradle dependency

2 Comments

1 Comment

Comments

Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

8 Answers 8

5 Comments

Maven dependency

Gradle dependency

2 Comments

1 Comment

Comments

Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related