0

I'm very new to Node.js. My goal is to develop d3 charts in HTML handling Data objects(Json Array) that have 100K plus rows using a CSV spreadsheet. I want to load the CSV file and convert it to JSON Array object in the server side so this will not crash IE11 when loading the CSV file in the front end. The CSV is a big file. When I create a node js file that creates a server and load/convert the CSV into JSON Array using require('csvtojson') module, I want to send the JSON Array object to my html so i can grab that object and plug into D3 charts. I'm trying to avoid having the "Not enough storage..." jquery error when using ajax. Is this possible? I'm trying really hard to find an example but no luck. Only sites i found is how to display the json as a string into the html page. Is there a way to grab the JSON Array object in the HTML page? This is the code i use to create a server and to make all my html pages display on the browser.

Updated: I added the csv() and it converts the data into jsonObject. So far its just getting the one json object which this is not the case. I just want to push the 'jsonData' object so i can grab that in the HTML page and plug it to d3. Is this possible? Please, need help. I cant use d3.csv() in html because the file is too large and the IE11 browser will crash. Thats why I do this in the server.

csv().fromFile(csvFilePath)
.on('json', (jsonObj) => {
    console.log(jsonObj);
    jsonData = jsonObj;
})
.on('done', (error) => {
    console.log('end');
}); http.createServer(function (req, res) {
    var q = url.parse(req.url, true);
    var filename = "." + q.pathname;

    fs.readFile(filename, function(err, html) {
        if(err) {
            res.writeHead(404, {'Content-Type': 'text/html'});
            return res.end('404 Not Found');
        }
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write(html);
        return res.end();
    });
}).listen(8080);

Is there a way to have the JSON Array object available when I load the HTML page and grab the object to plus into d3?

Also I tried this method but it didnt work which I open this question awhile back: Not enough storage error when using Ajax. Large response

Thank you. I tried explaining the best i can. Please need help..

7
  • d3.json can read json files from external sources into your d3 script. Or maybe I'm not understanding the question. Commented Nov 8, 2017 at 20:07
  • @RyanMorton no need for d3.json. I already have a function that handles and plugs the JSON Array data object into the d3 charts. The thing i'm trying to figure out is load the file from the server side, have the JSON array object ready in the server side, and on html page grab that object. Before i tried using d3.csv(...) in html but the IE11 browser can't handle amount of memory of that csv file and crashes. That's why I'm doing this on the server. Sorry, very new to Node.js and trying to find a solution without using DB Commented Nov 8, 2017 at 20:18
  • So you just need to stream the csv? stackoverflow.com/questions/33129677/… Commented Nov 8, 2017 at 21:44
  • @RyanMorton not sure what Stream does in node.js. I read about it and not sure how it works.. I have the jsonObject in the server side. I know how to parse the csv into the jsonObjects. Now i want to make that jsonObject available when i load the client side 'HTML' so i can grab that object. Is that possible? Commented Nov 8, 2017 at 22:18
  • I'm more confused now, so, all you need is to pass the JSON from the server to the client? You should rewrite your question. Have you tried using: stackoverflow.com/questions/34149101/… Commented Nov 8, 2017 at 23:11

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.