Parsing JSOUP table

Question

I want to parse a table using jsoup. I have tried to get the flight data but without success!

My code is:

try {
   doc = Jsoup.connect("a.html").timeout(13 * 1000).get();
   Element table = doc.select("table.arrive-depart-table tbody tr").first();
   Iterator<Element> iterator = table.select("td").iterator();
   Log.d("log", iterator.next().text());
}

and this is the html:

<table class="arrive-depart-table">
  <tbody>
    <tr>
      <td> string 1</td>
      <td> string 2</td>
      <td> string 3</td>
      <td> string 4</td>
      <td> string 5</td>
   </tr>
<tr>
<td> string 6</td>
...and more

Exception:

Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'org.jsoup.select.Elements org.jsoup.nodes.Element.select(java.lang.String)' on a null object reference

I am not able to parse the table flight.

Thank you

What is the question? Post url, expected output and current output. Also make sure that the html source contains the data with disabled JavaScript. — Frederic Klein
– Frederic Klein, Commented Oct 25, 2016 at 12:02
this arrive-depart-table class is fillup by javascript , so you need to use htmlunit driver or selenium — Parth Solanki
– Parth Solanki, Commented Oct 25, 2016 at 12:55

Frederic Klein · Accepted Answer · 2016-10-25 13:58:17Z

Using the background ajax call (check: chrome dev tools -> Network tab) and JSON.simple to parse the response, it is possible to get the data without JavaScript:

Example Code

try {
    String userAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36";
    String url = "https://www.maltairport.com/wp-content/themes/mia/flightsinfo.php?arrivalsDepartures_action=getArrivalsDepartures";
    String referer = "https://www.maltairport.com/passenger/flights-landing/arrivals-departures/";
    String host = "www.maltairport.com";

    Document doc = Jsoup.connect(url).userAgent(userAgent).header("Host", host).header("Referer", referer).ignoreContentType(true).get();

    JSONObject jsonObject = (JSONObject) new JSONParser().parse(doc.body().text());

    JSONArray arrivals = (JSONArray) jsonObject.get("arrivals");
    JSONArray departures = (JSONArray) jsonObject.get("departures");

    System.out.println("departures");

    for (Object object : departures) {
        jsonObject = (JSONObject) object;
        System.out.println("Flight: " + jsonObject.get("flightNumber") + "\n\t" + "To: " + jsonObject.get("airportName")
                + " SCH: " + jsonObject.get("scheduledTime") + " EST: " + jsonObject.get("estimatedTime") + " Status: " + jsonObject.get("remarks"));
    }

    System.out.println("\narrivals");

    for (Object object : arrivals) {
        jsonObject = (JSONObject) object;
        System.out.println("Flight: " + jsonObject.get("flightNumber") + "\n\t" + "To: " + jsonObject.get("airportName")
                + " SCH: " + jsonObject.get("scheduledTime") + " EST: " + jsonObject.get("estimatedTime") + " Status: " + jsonObject.get("remarks"));
    }

} catch (IOException | ParseException e) {
    e.printStackTrace();
}

Truncated Output

departures
Flight: FR 7243
    To: DUBLIN SCH: 15:15 EST: 15:36 Status: AIRBORNE
Flight: LS 650
    To: EAST MIDLANDS SCH: 15:15 EST: 15:33 Status: AIRBORNE    
[...]

arrivals
Flight: KL 3399
    To: AMSTERDAM SCH: 14:50 EST: 14:56 Status: LANDED
Flight: KM 395
    To: AMSTERDAM SCH: 14:50 EST: 14:56 Status: LANDED
[...]

Collectives™ on Stack Overflow

Parsing JSOUP table

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related