Rails Exporting to CSV without constructing CSV in memory

Question

Rails 2.3.5, Ruby 1.8.7.

In a prior question, there are a number of solutions to exporting data in CSV format, but some of them seem to construct the data in memory before sending. Is this wise when exporting large data sets? Do any of the solutions in the prior question avoid this.

Or is it impossible to avoid building a response without building the whole response locally, either in memory or in a temp file?

I would not be surprised if the latter was true, since if there is an error in the CSV generation, you might want to send an error message back instead, but I might be generating way too much data to want to generate the data in memory/on disk.

Austin Taylor · Accepted Answer · 2011-04-11 20:58:28Z

2

You can stream directly from the CSV library (FasterCSV in Ruby 1.8).

render :text => proc { |response, output|
    CSV.generate(output) do |csv|
      csv << ...
    end
}

You should also use find_in_batches if you are concerned about your memory footprint.

answered Apr 11, 2011 at 20:58

Austin Taylor

5,4771 gold badge26 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Rails Exporting to CSV without constructing CSV in memory

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related