1

I am downloading a large text file from a server to a phonegap app. The text is valid JSON in string form. I write the text data to the device's file system, then read it later. I fetch the text data using the filereader adn this seems to work fine. Its when I parse the string JSON that the problem occurs.

When I download a text file containing 2500 elements(records returned from a DB and returned in json format) the JSON.parse(stringJSON) function works fine and returns a JSON object, however if I increase the number of elements returned from the server to 3000 then I get no response, teh json output seems to be formatted correctly. It looks like a memory problem, the JSON data is at most 1.5MB, is that too much to parse using jquery's JSON.parse, with phonegap. I looked at this page and tried including parse_json instead of JSON.parse and it still returns nothing (I included the relevant script in my page).

var ft = new FileTransfer();

 ft.onprogress = function(progressEvent) {
    if (progressEvent.lengthComputable) {
        var perc = Math.abs(Math.floor((progressEvent.loaded / progressEvent.total) * 100));
        $loading.html(perc + " Loading...")
    } else {
        if($loading.innerHTML == "") {
            $loading.innerHTML = "Loading";
        } else {
            $loading.innerHTML += ".";
        }
    }
};

var dlPath = DATADIR_separate.fullPath + "/"+foldername_separate+".html";
ft.download("http://downloadURL", dlPath, function(){
    alert('saved');
    $loading.hide();
    remove_permanent_overlay();
}, function(){
    alert('fail');
    $loading.hide();
    remove_permanent_overlay();
});

then after all the file reader callbacks:

function readAsText_READ(file) {
var reader = new FileReader();

reader.onloadend = function(evt) {
    alert('loaded');//this fires, so filereader is working
    //alert(evt.target.result);//this alerts even with a large string
    //var parseTarget = JSON.parse(evt.target.result);//works for smaller strings fails with larger strings
    console.log('parsing now...');
    var parseTarget = json_parse(evt.target.result);//same result as JSON.parse
    console.log(parseTarget);
    //var parseTarget = $.parseJSON( evt.target.result );
    //var parseTarget = eval( "("+evt.target.result+")" );
    //alert(parseTarget);
    alert(parseTarget["879"]);
};

reader.readAsText(file);
//reader.readAsDataURL(file)
alert('reading file: '+file);

}

*Edit In phonegap's weinre debugger I get the following when I try to read the large json string.

parsing...
processMessage failed: Message: S01 File1238321490
s{"really long json... jsonetc..."}
processMessage failed: Error: [object Object]
processMessage failed: Stack: undefined

not quite sure what those error messages mean, I'll try find out...

Any help would be greatly appreciated.

6
  • Without showing an error message it is hard to guess. Commented Sep 18, 2013 at 10:11
  • sorry, I've edited my question with the output I got from the weinre console, not sure how helpful it is. Commented Sep 18, 2013 at 10:24
  • you should catch the error thrown by json_parse e.g. with try { var parseTarget = json_parse(evt.target.result); } catch( e ) { console.dir(e) } to see if it is the parser and what message it shows. Commented Sep 18, 2013 at 10:31
  • I tried the try catch, and got exactly the same error message, with no additional errors output to the console, so I guess that means that there's no problem with json_parse? Commented Sep 18, 2013 at 10:39
  • Without having the code in front it is not easy to tell. If calling the json_parse would not throw an error, then the problem seems to be somewhere else. Does probably your console.log(parseTarget); fail? Commented Sep 18, 2013 at 10:56

1 Answer 1

1

Just for some closure:

I reduced the size of the file to be downloaded and stupidly I forgot to escape the content being sent from the server to the mobile app, so it was receiving invalid JSON. This was hard to discover since the first few thousand entries in the JSON were valid, there were a few rogue double quotes later in the string.

I escaped the content from the server and easily managed to parse the JSON in a 300kb file. I'm pretty sure a 1MB file would also work fine.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.