JSoup seems to work with just about all URLs I've tried but this one gives me the 400 error.
String url = "http://localad.walmart.com?storeref=3008&forceview=y";
Response response = Jsoup.connect(url.replaceAll(" ", "%20"))
.method(Method.GET)
.userAgent("Mozilla")
.followRedirects(false)
.timeout(5000)
.data("pragma", "no-cache")
.execute();
Error I get is:
Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=400, URL=http://localad.walmart.com?storeref=3008&forceview=y&pragma=no-cache
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:449)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:424)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:178)