1

I am trying to web scrape some data so that I can use it in my application.

The site I am trying to get data off is yahoo but I am getting a FileNotFoundException when it's trying to stream the data in.

I have also set the IP address and port explicitly.

Would be really thankful if someone can tell me where I am going wrong.

I have posted the sample code as well.

parentUrl = "http://www.yahoo.com";
pageUrl = new URL(parentUrl);
System.out.println(parentUrl);

try {
    in = new BufferedReader(new InputStreamReader(pageUrl.openStream()));
} catch(Exception ex2) {
    ex2.printStackTrace();
}

while ((inputLine = in.readLine()) != null) {
    out.write(inputLine);
    in.close();
}

out.close();    
7
  • 1
    You need to show the code where you construct out. That'll be where the problem is. Commented Jan 31, 2012 at 6:04
  • 8
    Also note that you are calling in.close() inside your read loop. You'll never be able to read more than one line with this. Commented Jan 31, 2012 at 6:07
  • @CameronSkinner is right.. you just need to remove in.close() from your while loop. Commented Jan 31, 2012 at 6:12
  • @CameronSkinner - i removed that statement and pushed it outside the loop , its still giving me the same exception. its creating a file test.html in the desired location but with no data in it. Commented Jan 31, 2012 at 6:32
  • 1
    It would help if you showed us the actual exception stacktrace Commented Jan 31, 2012 at 6:40

1 Answer 1

1

The problem is in the initialization of out. You haven't shown us that code, but it will be something like:

OutputStream out = new FileOutputStream("non/existent/path/somefilename");

It's probably due to you using a relative path, so to help you debug it, I recommend you change it to:

File file = new File("non/existent/path/somefilename");
System.out.println(file.getAbsolutePath()); // start with this simple debugging
OutputStream out = new FileOutputStream(file);

My guess is that the path of the file is not where you think it is.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.