Android: get HTML from web page as String with HttpClient not working

Question

I am programming an app that uses HttpClient to connect to a web page (the aim is to be able to copy some of the HTML of a webpage into a String). I tried accomplishing this by using an HttpClient connection. This is the code I used:

public void getText() {
    final TextView contentView = (TextView) findViewById(R.id.textview);
   String url = "http://www.anandtech.com";
    /* An instance of this class will be registered as a JavaScript interface */ 
HttpClient client = new DefaultHttpClient();

HttpGet get = new HttpGet(url);
ResponseHandler<String> handler = new BasicResponseHandler();
try {
    String response = client.execute(get, handler);
    contentView.setText(response);
} catch (Exception e) {
    // TODO Auto-generated catch block
    contentView.setText(e.getMessage());
}

}

I found the source of the error by removing each statement of this block and then testing if the error still occurs and found that the error originated from the client.execute(get, handler). I have permission to use internet on AndroidManifest.xml, and I know the internet works anyway on the app because a WebView can load just fine. How can I make this work?

Also, I tried to get the String by injecting javascript into a webview url, with the following code:

MyJavaScriptInterface.class class MyJavaScriptInterface { private TextView contentView;

        public MyJavaScriptInterface(TextView aContentView)
        {
            contentView = aContentView;
        }
    }

MainActivity.java @SuppressWarnings("unused")

        public void processContent(String aContent) 
        { 
            final String content = aContent;
            contentView.post(new Runnable() 
            {    
                public void run() 
                {          
                    contentView.setText(content);        
                }     
            });
        } 
    } 

    webView.getSettings().setJavaScriptEnabled(true); 
    webView.addJavascriptInterface(new MyJavaScriptInterface(contentView),"INTERFACE"); 
    webView.setWebViewClient(new WebViewClient() { 
        @Override 
        public void onPageFinished(WebView view, String url) {
        webView.loadUrl("javascript:window.INTERFACE.processContent(
        document.getElementsByTagName('p')[0].innerHTML);");
        } 
    }); 

    webView.loadUrl("http://tabshowdown.blogspot.com");

Both the class and the processContent() method were defined in the Activity's void onCreate method, FIY. In this case the problem is the javascript isn't initiating the processContent() method, since a. If I try to initialize any other javascript command, e.g. document.write(), the webview reacts, so javascript isn't the problem. If I try to initiate the MyJavaScriptInterface and the processContent() method from anywhere else on the script, it works just fine. So it appears to me as if the javascript is having trouble initiating the method (even though other users reported this method to work).

Can someone help me get either of these methods to work? Thanks in advance

user1888162 · Accepted Answer · 2013-06-08 01:38:12Z

1

This should work:

public class MainActivity extends Activity{
    private TextView contentView;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(your layout);
        contentView = (TextView) findViewById(R.id.textview);
        DownloadTask task = new DownloadTask();
        task.execute(url);
    }
    private class DownloadTask extends AsyncTask<String, Void, String>{

            @Override
            protected String doInBackground(String... urls) {
                HttpResponse response = null;
                HttpGet httpGet = null;
                HttpClient mHttpClient = null;
                String s = "";

                try {
                    if(mHttpClient == null){
                        mHttpClient = new DefaultHttpClient();
                    }


                    httpGet = new HttpGet(urls[0]);


                    response = mHttpClient.execute(httpGet);
                    s = EntityUtils.toString(response.getEntity(), "UTF-8");


                } catch (IOException e) {
                    e.printStackTrace();
                } 
                return s;
            }

            @Override
            protected void onPostExecute(String result){
                contentView.setText(result);

            }
        }
}

edited Jun 8, 2013 at 1:38

answered Jun 8, 2013 at 0:38

user1888162

1,74521 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user2465260 Over a year ago

Can the class be a local class inside the OnCreate method? or does it have to be on a separate .java file? Also, Eclipse says that the "protected Document doInBackground()" is "incompatible with AsyncTask.doInBackground"

user2465260 Over a year ago

Never mind, I got it to work.... strange, though.... it appears to only have gotten the <html> tag

user2465260 Over a year ago

nevermind again.... thanks a lot btw, I wasn't finding a working method anywhere.

Shajeel Afzal Over a year ago

It is returning everything including javaScript, How can we modify this trick to get HTML only. Thanks.

Collectives™ on Stack Overflow

Android: get HTML from web page as String with HttpClient not working

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related