0

I writing downloader app. I want to only use java Socket to request a file. So I write in my socket just as HTTP protocol rules. My app creates a connection and after reading headers, uses read() method of InputStream of my socket. Everything goes well. sometimes connection may lost. but I store which byte I was reading, so again it creates a new Socket with HTTP Ranged GET and continue its work. But when download is about to finish, I mean when less than 10 KB remains, All Connections will lost and it again(as scheduled) try to open new Socket and Continue its work. It completely reads headers of response but before reading any byte of body, read() method returns -1 and Again and Again it tries to Open a new Socket and read() remaining bytes, but the problem persists. The point is every time response headers can completely be read. and I see the Content-Length: field of response header is exactly the remaining bytes of file. I forgot to mention: my code has a problem because I check many file from many servers and result is the same. here is the Code:

// Some fields:
int state;
long start, current, end;

// in a thread:
while (state != FINISHED) {
    if (state == DOWNLOADING) {
        try {
            // fill a new socket with Ranged GET [current, end]
            Socket s = initConnection();
            InputStream in = s.getInputStream();
            int readNo = 0;
            FileOutputStream out = getTempFile();
            byte[] buffer = new byte[1024];
            // read response headers successfully and prints them, request range is OK. a sample of its print is at the end of page
            readHeaders(in);
            while (state == DOWNLOADING && (readNo = in.read(buffer)) != -1) {                      
                current += readNo;
                out.write(buffer, 0, readNo);
            }
            if (readNo == -1) {
                // at nearly end of download always print this and values never changes, where usually they have 3000 byte difference
                System.out.println("**************> (" + current + " - " + end + ")");
            }
            if (currentByte == endByte) {
                state = FINISHED;
                //mergeParts();
                // code never reaches here
                dlInfo.checkAllPartsFinished();
            }
            out.flush();
            out.close();
            s.close();
        } catch (Exception e) {
            e.printStackTrace();
            state = ERROR;
            error = e.getMessage();
            errorRetry++;
        }
    } else if (state == PAUSED) {
        // ...
    } else ...
    }
}

a sample of response header at the end of file where nothing changes:

HTTP/1.1 206 Partial Content
Date: Mon, 21 May 2012 14:34:27 GMT
Server: Apache
Last-Modified: Sat, 21 Apr 2012 02:16:20 GMT
ETag: "4006d32e-f691e0-4be26fda00500"
Accept-Ranges: bytes
Content-Length: 7859
Content-Range: bytes 2012041-2019899/16159200
Connection: close
Content-Type: application/octet-stream

**************> (2012041 - 2019899)

I don't know what problem is but whatever it is, it occurs at nearly the end of stream. I'm completely confused, after many hours spending time. I would be grateful for Any help!

thanx

3
  • Whad does readHeaders do? Are you sure that it doesn't read more than just the headers? Does it use a buffer of some kind? Commented May 21, 2012 at 14:42
  • In readHeaders I create a BufferedReader from socket input stream and read exactly till body. If it was not exact it would print some of body in Console!! Commented May 21, 2012 at 14:51
  • 2
    @MHM: that's your problem right there! A BufferedReader will read into a buffer from which you can get stuff. That means that more than the headers will have been read from your InputStream when readHeaders returns. This means that the data you are missing is at the beginning, not at the end. Commented May 21, 2012 at 15:02

1 Answer 1

3

Are you layering a Buffered Reader/Stream on top of the InputStream in your readHeaders() method? my guess is that you are doing this, and that this buffered stream is reading more of the InputStream than you expect (since it is buffered). these bytes are then lost when you return from the readHeaders() method.

UPDATE:

just saw your last comment. that is exactly your problem. the BufferedReader is consuming part of the body bytes.

Sign up to request clarification or add additional context in comments.

4 Comments

Thanks mate, you've got something, let me check it!
Reading the headers correctly is one of the trickiest parts of implementing the HTTP protocol in Java.
Hint: it's probably best to read into a byte-array and find the \r\n\r\n (CR LF CR LF) sequence (and empty line) and then convert the part before that into a String (that's your headers) and everything after that is the beginning of your data.
Thanks again mate,, I even don't think of such problem!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.