3

I am trying to export different types of data from my DB to a CSV file in Nodejs and express. I have tried several libraries so far and nothing seems to work as I expected for many varied reasons.

How can I work this out? what should I know in order to be able to export all the data I want to CSV file? and How can I force my browser to do it?

Thanks

1

1 Answer 1

5

So, after a lot of struggling, I will share my main insights, that are not that obvious to whom who are making their first steps in web development.

Exporting to CSV can be divided into two main steps: 1. Arranging your Data into CSV structure/model. 2. Exporting the data / make it be downloaded on the client side.

So I'll break it down. The first step - Arranging your Data into CSV structure/model: To get your data into CSV structure, most likely you can find a library that takes the data you want to export and format it into CSV. If your data model is as complexed as mine, you will have to create a custom function. Either way, that shouldn't be too complicated. an example of such function that I used:

// The function gets a list of objects ('dataList' arg), each one would be a single row in the future-to-be CSV file
// The headers to the columns would be sent in an array ('headers' args). It is taken as the second arg
function dataToCSV(dataList,headers){
    var allObjects = [];
    // Pushing the headers, as the first arr in the 2-dimensional array 'allObjects' would be the first row
    allObjects.push(headers);

    //Now iterating through the list and build up an array that contains the data of every object in the list, in the same order of the headers
    dataList.forEach(function(object){
        var arr = [];
        arr.push(object.id);
        arr.push(object.term);
        arr.push(object.Date);

        // Adding the array as additional element to the 2-dimensional array. It will evantually be converted to a single row
        allObjects.push(arr)
    });

   // Initializing the output in a new variable 'csvContent'
    var csvContent = "";

    // The code below takes two-dimensional array and converts it to be strctured as CSV
    // *** It can be taken apart from the function, if all you need is to convert an array to CSV
    allObjects.forEach(function(infoArray, index){
      var dataString = infoArray.join(",");
      csvContent += index < allObjects.length ? dataString+ "\n" : dataString;
    }); 

    // Returning the CSV output
    return csvContent;
}

Now, the second step - Exporting the data: In order to export the data, after examining few options, I found that the most convenient (for me), was to sent the data over the HTTP header and make the browser to download the file and parse it as CSV. That I made with the following code:

//this statement tells the browser what type of data is supposed to download and force it to download
    res.writeHead(200, {
        'Content-Type': 'text/csv',
        'Content-Disposition': 'attachment; filename=*custom_name*.csv'
    });
// whereas this part is in charge of telling what data should be parsed and be downloaded
    res.end(dataToCSV(dataList,["ID","Name","Date"]),"binary");

In conclusion, I made this post so others won't struggle like I did when it comes to exporting CSV using nodejs and express. If you find any errors, or you think that some of the written above should be explained more thoroughly, please let me know and I'll make the necessary changes.

Kind Regards.

Sign up to request clarification or add additional context in comments.

4 Comments

Just a quick next step: depending on how you request the file, you may also need download the response. You can either use this package: github.com/kennethjiang/js-file-download Or pop your head in the source code and write it yourself (it's only 30 lines)
I was in your shoes today, so thank u very much for this detailed answer
@sale108, What will happen if large data?
@user1149244 - didn't need it back then. obviously if it's to large to fit in memory, you'll need to user other methods (maybe streaming it) and re-assemble on the receiver side

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.